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PREFACE 


This  report  documents  the  results  of  a  comprehensive  study  to  create  a  concurrent 
communication  feedback  mechanism  as  part  of  a  logistics  research  and  development 
program  titled  Communications  Assessment  Pilot  (CAP),  (Contract  Number  F41 624-97- 
D-5002)  managed  by  the  Air  Force  Research  Laboratory,  Logistics  Sustainment  Branch 
(AFRL/HESS),  at  Wright-Patterson  AFB,  OH.  The  primary  goal  of  the  project  was  to 
develop  and  pilot  a  manual  mechanism  for  change  agents  to  monitor  the  effectiveness  of 
communications  across  all  levels  and  functions  in  an  organization  on  an  ongoing  basis. 
The  pilot  determined  the  feasibility  of  developing  an  automated  system  that  will  allow 
managers  to  determine  the  effectiveness  of  selected  communication  efforts.  This 
monitoring  allows  for  near  real-time  corrections  in  communication  choices  when 
effectiveness  is  being  compromised.  The  intent  is  that  this  mechanism  will  become  a 
permanent  management  tool  for  change  agents  and  managers  at  the  participating 
organizations. 
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1 .0  PROJECT  OVERVIEW 


This  report  describes  the  tasks  performed  in  the  pilot  development  of  a  concurrent  communication  feedback 
mechanism.  Change  efforts  currently  underway  at  two  Department  of  Defense  (DoD)  organizations:  Wamer-Robins 
Air  Logistics  Center  (WR-ALC)  and  the  Defense  Reutilization  and  Marketing  Service  (DRMS)  provided  the 
research  context.  This  project,  referred  to  as  the  Communication  Assessment  Pilot  (CAP),  complements  the 
Readiness  Assessment  and  Planning  Tool  Research  ( RAPTR )  project,  sponsored  by  the  Air  Force  Research 

Laboratory  (AFRL/HESS).  CAP  took  advantage  of  change  agents  within  both  organizations  to  assist  in  definition 
testing,  and  deployment. 


This  project  developed  and  piloted  a  manual  mechanism  for  change  agents  to  monitor  the  effectiveness  of 
communications  across  all  levels  and  functions  at  the  participating  organizations  on  an  ongoing  basis.  The  pilot 
determined  the  feasibility  of  developing  an  automated  system  that  will  allow  managers  to  determine  the  effectiveness 
of  selected  communication  efforts.  This  monitoring  allows  for  near  real-time  corrections  in  communication  choices 
when  effectiveness  is  being  compromised.  The  intent  is  that  this  mechanism  will  become  a  permanent  management 
tool  for  change  agents  and  managers  at  the  participating  organizations.  Developing  and  deploying  a  more  automated 
version  of  this  tool  is  a  follow-on  project. 

Individual  communication  efforts,  referred  to  within  this  document  as  communication  events,  can  include  briefings, 
memos,  reports,  or  any  other  discrete  communication  that  has  a  defined  purpose  and  a  defined  audience.  For  each  ’ 
event  monitored,  the  initiator  of  that  communication  was  asked  a  series  of  questions  about  the  goals  for  the 
communication,  the  timing  of  the  communication,  and  the  intended  audience.  The  goal  was  to  reduce  the  questions 
asked  about  a  communication  event  to  a  standard  communicator  input  form.  The  answers  to  these  questions  were  to 
be  used  to  create  a  custom  feedback  form  for  that  event. 

This  defined  audience  comes  from  a  variety  of  levels,  within  management  and  line,  military  and  civilian.  These 
individuals  are  referred  to  in  the  text  below  as  receivers .  The  idea  within  CAP  was  to  identify  the  intended  audience 
within  the  organizational  unit  for  the  pilot  and  then  categorize  them  in  some  way  that  is  meaningful  to  the 
management  of  that  organization.  Then,  for  selected  communication  events,  individuals  from  these  categories  were 
sampled  to  get  feedback  on  that  event.  This  reflects  the  fact  that  different  categories  of  employees  can  hear  and 
understand  things  differently.  This  sampling  sought  to  highlight  these  differences.  The  sample  was  asked  to 
complete  the  feedback  form  for  that  event.  The  data  was  collected  and  analyzed,  with  feedback  compared  to  the 
intent  of  the  person  that  initiated  the  communication. 

In  the  pilot,  the  communication  goals  were  used  to  manually  create  a  feedback  form.  People  were  sampled,  data 
gathered,  and  the  questionnaire  data  coded  by  hand.  The  idea  during  the  pilot  is  to  create  a  procedure  that  could,  in  a 
subsequent  effort,  be  automated  in  a  computer  program. 

Conducting  the  pilot  at  two  different  organizations  provided  several  benefits.  By  phasing  the  technical  activities, 
learning  from  one  site  can  immediately  be  applied  that  learning  to  work  at  the  other  site.  This  also  resulted  in  some 
economies  of  scale  in  the  research  effort  by  developing  many  project  procedures  generally  enough  to  be  applied  at 
both  sites.  However,  there  were  enough  differences  between  the  sites  that  many  products  could  not  be  directly 
applied  at  both  sites.  For  example,  the  initial  data  collection  at  each  site  required  data  collection  instruments.  These 
instruments  had  some  common  elements  but,  due  to  the  differences  in  the  sites,  they  also  contained  site-specific 
items.  It  was  anticipated  that  the  feedback  collection  mechanisms  would  be  substantially  similar  but  their  means  of 
use  would  most  likely  be  different.  Finally,  the  specification  for  the  software  system  could  also  have  some  site- 
specific  differences.  Overall,  however,  having  two  pilot  sites  resulted  in  a  more  robust  result  that  can  be  more  easily 
applied  at  other  DoD  organizations. 

The  project  team  for  this  pilot  consisted  of  staff  from  the  Environmental  Research  Institute  of  Michigan  (ERIM)  and 
an  independent  consultant  who  specializes  in  communications  research  and  practices. 
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3.0  OBJECTIVE 

The  objective  of  the  CAP  Project  was  to  define,  develop,  and  test  a  mechanism  to  provide  near  real-tone  feedback  on 
the  effectiveness  of  communication  within  an  organization.  This  mechanism  lays  the  foundation  f°r  ^e 
development  of  a  software  tool  that  could  semi-automatically  collect 

was  conducted  within  one  WR-ALC  PD/function  and  a  selected  unit  of  DRMS.  The  CAP  Project  was  mo  y 

two  key  issues: 

.  Considerable  evidence  exists  that  communication  effectiveness  is  currently  less  than  optimal  at  the  P^ipatmg 
organizations.  At  WR-ALC,  for  example,  this  is  potentially  contributing  to  individual  and  group  resistan 

assessment? 


4.0  APPROACH 

Specific  tasks  and  deliverables  associated  with  this  project  are  identified  in  the  following  paragraphs. 


4. 1  Communications  Assessment 

4  11  Select  Organizational  Units 

At  WR-ALC  it  took  some  discussion  to  select  the  appropriate  unit.  The  original  request  for  “communication 
isSce”  caml  from  the  F-15  Weapon  System  Support  Center  (WSSC),  a  new  organizational  unit  created  to 
implement  the  Aircraft  Repair  Enhancement  Program  (AREP).  Consequently,  they  were  chosen  as  bo*  toe 
organizational  and  change  focus.  While  the  original  SOW  indicated  a  focus  °n  one  umt  ^tton^h  targ 
organization,  the  project  team  agreed  to  include  both  the  F-15  WSSC  and  the  C-141  WSSC  at  WR-ALC. 

WSSC  has  less  than  100  people 

The  kickoff  meeting  at  DRMS  was  conducted  on  4/29/98.  At  this  session  it  was  decided  that  the  effort  at  DRMS 
““S  ™«,  a  new  par,  of  to  oration  created  m 

Management  initiative.  Enterprise  Management  is  a  systematic  approach  to  making  public/pnvate 
public  organizations.  It  can  include  elements  of  A-76  for  specific  decisions. 

There  are  approximately  120  Marketplace  staff  at  HQ  DRMS  and  approximately  500  Pe°P1^^^dst^;atd0nsaff 
work  at  Defense  Reutilization  and  Marketing  Offices  (DRMOs)  that  are  usually  collocated  with  DoD  installations. 
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These  DRMOs  accept  material  for  reutilization/transfer/donation  (R/T/D).  These  operations  perform  several  very 
different  functions,  including  retailing,  performing  recycling  actions,  and  demolishing  certain  items. 

4.1 .2  Meet  with  Unit  Management 

The  premise  of  CAP  is  that  management  communicates  across  all  levels  of  the  organization  using  multiple 
mechanisms,  as  illustrated  in  Figure  1.  As  part  of  our  investigation,  the  project  team  had  to  identify  the  types  of 
mechanisms  and  their  respective  audiences. 


Figure  1  -  Management  communication  mechanisms 


Subject  to  their  availability,  the  project  team  scheduled  meetings  with  the  management  of  the  selected  organizational 
units  to  discuss  the  project  concept  and  to  gather  more  in-depth  information.  Information  to  be  gathered  included: 

•  What  receivers,  i.e.,  employees  of  different  classes  within  the  target  organizational  units,  should  currently  know 
about  specific  change  efforts  through  interviews  of  communicators  and  content  analysis  of  recent 
communication  events. 

•  The  anticipated  frequency  and  nature  of  communication  events  and  how  often  monitoring  events  may  be 
required 

•  The  desired  sampling  characteristics,  i.e.,  what  criteria  would  managers  of  the  organizational  units  like  to  use  to 
select  respondents  and  view  summary  statistics  about  their  communication  effectiveness. 

This  project  could  not  be  successful  without  the  participation  of  organizational  unit  personnel.  These  “change 
agents”  were  identified  with  the  participation  of  RE  at  WR-ALC  and  DRMS  management  at  DRMS.  Change  agents 
are  personnel  that  are  currently  involved  in  change  efforts  who  may  be  willing  to  provide  guidance  during 
development  and  assistance  during  deployment.  At  WR-ALC,  the  C-141  WSSC  Chief  (WR-ALC/LFPW)  was  a 
constant  supporter  of  our  efforts,  providing  time  and  inputs  in  the  face  of  an  increasing  workload.  At  DRMS,  there 
were  several  points  of  contact  (POCs)  during  the  different  phases  of  the  project.  For  the  most  part,  their  efforts 
moved  the  project  forward.  However,  changing  POCs  five  times  did  cause  the  project  team  to  expend  time  and 
effort  to  bring  these  new  collaborators  up  to  speed. 

Before  the  site  visits,  the  project  team  developed  a  detailed  set  of  objectives  for  the  site  visits,  included  here  as  Appendix  B. 
Working  from  this  blueprint  helped  keep  the  visits  on  track  and  provided  a  checklist  that  ensured  the  project  team  obtained 
the  desired  information. 
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At  WR-ALC,  the  team  met  with  the  WSSC  chief  for  each  organization.  Previous  experience  with  AREP  and  these 
particular  people  simplified  this  step  at  WR.  The  team  already  had  a  significant  amount  of  archival  information  that 
was  supplemented  during  the  visit.  After  the  kickoff  briefing  at  DRMS,  the  team  spent  some  time  with  senior  HQ 
DRMS  staff,  discussing  issues  and  deciding  on  the  appropriate  organization  for  this  project. 

An  important  lesson  for  projects  of  this  type  is  that  people  undergoing  change  tend  to  be  very.  It  is  difficult  to  get 
active  participation  and  timely  responses  to  information  requests.  To  the  greatest  extent  possible,  plan  your 
interactions  and  data  collection  so  that  those  assisting  you  can  do  everything  you  need  as  part  of  your  structured 
meeting. 

4.1.3  Develop  Instruments  for  Each  Organizational  Unit 

The  project  team  conducted  a  literature  review  to  identify  specific  instruments  for  specific  issues  (see  Appendix  C). 
The  final  survey  was  constructed  around  two  existing  surveys: 

•  A  36-item  organizational  culture  instrument  developed  by  Glaser  et  al  (1987). 

•  A  23-item  organizational  communication  instrument  developed  by  Roberts  and  O’Reilly  (1974). 

Some  minor  modifications  were  made  to  the  Roberts  and  O’Reilly  instrument.  For  example,  relevant  response 
options  were  added  for  some  questions.  Since  trust  is  an  important  issue  at  both  organizations,  several  items  were 
added  about  trustworthiness  and  believability  of  information.  Some  demographics  were  also  included: 

•  Location 

•  Grade 

•  Union/non-union 

•  Years  in  present  location  (WR-ALC/DRMS) 

•  Years  in  present  position 

Both  DRMS  and  the  WR-ALC  WSSCs  are  evolving  to  information-based  organizations  that  will  require  most 
employees  to  be  computer-literate.  At  the  WSSCs,  most  staff  members  have  their  own  computer  and  Internet 
electronic  mail  address.  At  DRMS,  becoming  an  information-based  organization  is  one  of  the  tenets  of  their  change 
efforts.  Members  of  the  project  team  visited  two  DRMOs  as  part  of  this  effort,  conducting  unstructured  interviews 
with  DRMO  management.  Different  DRMOs  have  taken  different  approaches  to  email  deployment.  At  some 
locations,  most  people  that  need  to  use  a  computer  have  one  that  has  access  to  the  outside  world.  At  others,  just  the 
Chief  has  an  external  email  address,  with  some  locations  having  local  area  networks  (LANs)  for  internal  distribution. 
Access  is  just  one  issue.  Training  and  support  are  equally  problematic.  During  an  organization-wide  training  course, 
DRMS  employees  were  informed  that  computer  literacy  was  a  goal  for  all  employees.  Some  needed  to  be  brought 
up  to  speed  on  basic  applications,  while  others  required  training  on  new  business  systems.  For  these  reasons,  the 
project  team  decided  to  include  a  section  in  each  survey  on  computing  issues.  Questions  areas  included: 

•  Do  you  have  a  computer? 

•  Self-assessment  of  skill  level  -  Expert,  Intermediate,  Average,  Below  Average 

•  Percentage  of  work-time  using  the  computer 

•  Is  your  computer  connected  to  the  Internet? 

•  For  DRMO  chiefs  and  WSSC  chiefs,  the  team  wanted  to  understand  the  deployment  of  computing  technology, 
i.e.,  how  many  people  do  you  have,  how  many  computers  (assigned  to  one  person  or  shared)  are  deployed,  and 
how  many  of  them  have  access  to  the  Internet  and  email. 


#  of  People 

Computers 

Email 

Internet 

Personal 

Shared 

Access 

•  Training  -  amount,  perceived  adequacy 
•  Applications 
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•  Legacy  systems  (and  their  replacements) 

•  Internet 

•  Support  for  used  systems  -  amount,  perceived  adequacy 

Earlier  questionnaire  sections  provided  information  about  the  people  and  the  environment  in  which  they  work.  The 
final  part  of  the  communication  baselining  instrument  was  for  facts  testing.  To  baseline  someone’s  knowledge  about 
an  organization  or  change  effort,  you  need  to  come  up  with  a  set  of  relevant  facts  about  the  organization  and  the 
change  to  test.  Much  of  this  information  was  initially  derived  from  archival  information.  For  DRMS,  archival 
materials  included: 

•  Issues  of  the  "EM"  News,  a  publication  of  the  DRMS  Communications  Team  for  Enterprise  Management.  34 
issues  published  between  23  September  1996  and  July  28  1998 

•  4  issues  of  “DRMS  World”,  a  DRMS  newsletter,  published  in  1997  and  1998. 

•  Employee  manuals. 

•  DRMS  State  of  the  Command  briefing  dated  10/1/97. 

•  DRMS  Business  Education  booklet,  “The  Challenge:  The  Choice  to  Change”,  dated  1997. 

•  DRMS  WWW  sites,  both  public  and  their  Intranet 

For  WR-ALC,  information  reviewed  included: 

•  Briefings 

•  AREP  AFMC  Command  and  Team  Conference 

•  AREP  Importers  Meeting 

•  AREP  Progress  Report  to  the  WR-ALC  commander 

•  AREP/DREP  Buying  Support 

•  The  Day  of  the  Fixer  is  Dawning... Ready  or  Not 

•  AREP  documentation 

•  “Re-engineering  a  Defense  Depot  to  Implement  Lean  and  Synchronous  Logistics  Workflows” 

•  The  WSSC  manual  and  supporting  documentation 

•  Weapon  System  Support  Center  (WSSC)  for  the  Aircraft  Repair  Enhancement  Program 

•  AREP  Manual  -  AREP  Process  Description 

The  result  was  two  “facts  databases”,  Excel  spreadsheets  with  the  following  columns: 

•  Fact  -  a  listing  of  the  potential  facts;  often  quotes  were  taken  from  the  original  source 

•  Source  -  archival  document  where  the  fact  was  located 

•  Who  -  the  document  reviewer. 

•  Comment  -  any  comment  about  the  facts,  e.g.,  if  there  was  more  detail  in  the  source,  related  facts,  who  should 
know  a  fact,  etc. 

DRMS  and  WR-ALC  staff  members  reviewed  the  fact  lists  and  provide  a  prioritized  list  of  approximately  15-20 
facts.  Of  course,  to  test  knowledge  you  can  not  just  ask  questions  that  are  all  true.  Some  plausible  “non-facts”  should 
be  included  as  options.  Therefore,  when  they  reviewed  the  “truths”,  they  were  asked  to  also  provide  as  many 
misconceptions  about  their  organization  and  the  change  effort  as  they  could.  Using  this  process,  the  final  section  of 
the  data  collection  instrument  included  multiple  choice  and  true/false  questions.  The  baselining  questionnaires  are 
included  in  Appendix  D  and  E. 

Building  the  fact-based  section  proceeded  smoothly  at  WR-ALC.  One  reason  for  this  was  the  team’s  understanding 
of  AREP  and  the  WSSC.  At  DRMS,  it  took  many  more  meetings,  emails,  and  telephone  calls  to  make  this  happen. 

4.1.4  Pilot  Instruments 

At  WR-ALC,  the  questionnaire  was  administered  to  the  WSSC  chiefs.  At  DRMS,  the  survey  was  tested  with  two  former 
DRMO  chiefs  who  had  recently  been  assigned  to  HQ.  Other  HQ  staff  reviewed  the  instrument  through  two  review  cycles. 
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4.1.5  Collect  Data  for  Each  Organizational  Unit 

At  WR-ALC,  all  F-15  and  C-141  WSSC  staff  was  requested  to  participate  in  the  baselining  survey. 

At  DRMS,  it  was  decided  to  focus  on  DRMOs  in  the  Continental  United  States  (CONUS).  The  DRMS  “chain  of 
command”  goes  from  HQ  DRMS  to  Zone  Managers,  staff  members  that  are  responsible  for  groups  of  DRMOs  in  a 
region.  A  DRMO  Chief  manages  these  DRMOs  or  “stores”.  Depending  on  the  size  of  the  organization,  there  could 
be  up  to  4  or  5  first  line  supervisors. 

There  are  several  communication  issues  worthy  of  study.  There  could  be  regional  effects  based  on  Zone  Manager 
communication  practices.  Organization  size  also  affects  communication.  To  investigate  these  issues,  the  project 
team  decided  to  survey  34  DRMOs  in  the  East  and  West  regions.  The  team  worked  with  HQ  DRMS  to  select  17 
DRMOs  from  each  region  that  were  “small”,  “medium”,  and  “large”.  Since  the  focus  was  on  the  Marketplace  unit, 
the  sample  included  staff  members  who  regularly  communicated  with  this  unit.  At  HQ,  this  included  all  Marketplace 
staff.  In  the  field,  the  DRMO  Chiefs,  the  sales  writers,  and  environmental  protection  people  were  included. 

HQ  DRMS  staff  identified  the  small,  medium,  and  large  DRMOs  in  the  East  and  West  regions.  The  project  team  put 
together  a  mailing  list  from  the  DRMS  phone  book  (dated  June  1997)  for  review  and  editing  by  the  Marketplace 
staff.  They  also  provided  numbers  on  how  many  staff  of  each  position  type  worked  at  each  DRMO.  To  build 
awareness  of  our  plans,  the  survey  effort  was  promoted  at  the  weekly  Zone  Managers  teleconference. 

The  team  used  this  respondent  information  to  send  a  packet  to  each  DRMO.  The  Chief  and  each  staff  member 
(positions  types  1104,  1 107,  028,  and  029  only)  received: 

•  A  cover  letter  describing  the  survey  effort. 

•  A  blank  copy  of  the  communication  baselining  survey  coded  only  by  location  (DRMOs  1-34,  HQ  35). 

•  A  self-addressed  stamped  envelope. 

The  survey  administration  instructions  to  the  field  led  to  some  confusion  about  whether  DRMO  Chiefs  should 
participate  in  the  survey.  As  part  of  the  callbacks  conducted  by  the  team,  the  telephone  respondent  was  told  about 
surveys  processed  to  date  from  their  location.  (Some  of  the  early  data  analysis  showed  some  other  issues 
necessitating  calls  to  the  DRMOs.)  If  they  had  not  already  responded  to  the  survey,  they  were  asked  the  objective 
questions  about  their  computing  environment  (numbers  107  through  111).  Asa  result  of  these  calls,  incorrect 
DRMO  Chief  contact  information  on  the  original  spreadsheet  was  corrected. 

All  surveys  were  returned  directly  to  the  project  team 

There  were  several  survey  administration  lessons  learned.  Individual  response  envelopes  seemed  to  work  well. 
Several  questions  asked  respondents  to  apportion  a  whole  amount  (communication,  effort,  etc.)  among  several 
options  that  were  to  total  100%.  It  was  clear  from  the  data  collected  that  respondents  sometimes  do  not  assure  their 
totals  sum  to  100.  Finally,  survey  instructions  about  conditional  sections,  i.e.,  only  some  respondents  need  to  reply, 
needed  to  be  clearer  and  more  emphatically  displayed. 


4.1.6  Analyze  Data  for  Each  Organizational  Unit 

The  surveys  were  categorized  by  site  code  and  marked  with  a  unique  survey  ID  number.  The  analysis  was  conducted  using 
Microsoft  Office  tools. 

Since  one  goal  of  the  project  was  to  define  how  these  surveys  could  be  conducted  automatically,  the  project  team 
used  the  analysis  and  reporting  task  as  an  opportunity  to  try  out  different  techniques.  For  the  WR-ALC  data,  the  site 
reports  (F-15,  and  C-141)  were  created  by  linking  analyses  from  an  Excel  spreadsheet  to  a  blank  survey  in  Microsoft 
Word.  A  blank  survey  was  annotated  with  the  summary  statistics  from  the  data  analysis.  This  annotated  survey 
became  part  of  the  reporting  to  WR-ALC.  While  this  was  effective,  it  used  significant  computing  overhead  that 
taxed  a  150  MHz  PC  to  frequent  failure.  The  project  team  also  wrote  a  report  that  described  the  survey  process  and 
results  more  fully.  Our  reports  are  included  as  Appendix  D  and  E. 
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For  DRMS,  the  team  created  a  large  Excel  workbook  with  sheets  for  data,  each  analysis,  and  a  results  summary. 
Pasting  the  raw  data  into  the  analysis  sheets  produced  all  of  the  necessary  summary  statistics.  These  graphs  and  data 
were  exported  to  Word  to  produce  the  report. 

The  team  delivered  the  results  to  both  organizations.  Team  members  visited  WR-ALC  and  met  with  both  WSSC 
Chiefs.  (At  this  meeting,  the  F- 15  WSSC  decided  that  they  did  not  have  the  staff  resources  to  continue  supporting 
the  project.  It  was  agreed  that  the  project  would  push  on  with  C-141  and  keep  the  F-15  WSSC  management  updated 
on  the  results.)  At  DRMS,  DRMS  management  was  briefed  on  12/15/98  at  HQ  DRMS  in  Battle  Creek,  MI.  To  get 
the  DRMS  results  out  to  the  field,  the  project  team  worked  with  the  HQ  Public  Affairs  staff  to: 

•  Write  an  article  about  the  survey  results  for  the  Enterprise  Management  newsletter  that  is  distributed  to  the 
entire  organization. 

•  Publish  WWW  versions  of  the  survey,  results  report,  and  the  spreadsheet  of  the  summary  statistics  on  the 
DRMS  Intranet. 


4. 2  Planning  and  Development 

4.2.1  Review  Available  Assessment  Instruments 

The  instrument  described  in  Davis  (1953a,  b)  was  our  starting  point.  The  project  team  also  reviewed  other 
possibilities,  such  as  those  described  in  Rubin  et  al  (1994).  The  results  of  the  literature  review  are  included  in 
Appendix  C.  After  this  review,  the  team  determined  that  the  ECCO  instrument  described  by  Davis  was  the  best 
alternative  for  this  pilot  test. 


4.2.2  Select  Change  Agents 

At  DRMS,  the  project  team  started  with  the  Executive  Vice  President  of  the  Marketplace  unit,  who  appointed  one  of 
his  staff  as  our  functional  point  of  contact  (POC).  At  WR-ALC,  the  team  worked  directly  with  the  WSSC  Chiefs. 


4.2.3  Develop  Communicator  Form 

The  questions  and  format  of  the  communicator  form  were  dictated  by  the  structure  of  the  feedback  form.  Simplicity  was  the 
goal. 


Corxnunicowr  _ 


i  .£  l<cA«|hte///CWi^DOWS/D£StTOP/>o.v^ 


Enter  top  3  messages 


.  Goal  #1 
Goal  #2 
i  Goal  #3 


Select  audience 


Chiefs,  Union 


>  ..DoatfTwtOotw  . .  ] 


Figure  2  -  Simple  communicator  form  to  be  built  around  feedback  form 
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This  was  one  area  that  was  difficult  to  implement.  Since  effective  communication  was  the  issue,  trying  to  use  a 
closed-ended  form  to  collect  communication  goals  and  audiences  was  problematic.  As  it  turned  out,  there  was  a 
significant  amount  of  review  and  revision  of  the  goals  for  each  survey.  Follow-on  research  efforts  should  develop 
some  training  on  this  process,  based  on  realistic  examples  taken  from  the  participating  organization.  In  a  true 
empirical  effort,  researchers  would  be  concerned  about  affecting  the  outcome  of  the  “experiment”  by  changing  the 
subject’s  behavior  before  measurement.  Here,  the  goal  was  to  improve  communication  effectiveness.  Effort  a  priori 
and  measurement  ex  ante  are  both  fair  game  as  long  as  the  result  is  achieved. 

4.2.4  Develop  Feedback  Form 

Computer  literacy  issues  at  WR-ALC  forced  the  team  to  use  paper-based  surveys.  On  the  other  hand,  the  geographic 
dispersion  of  DRMS  practically  forced  the  team  to  use  the  WWW  for  their  surveys.  A  password-protected  site  was  created 
on  ERIMs  secure  server  for  the  project.  An  HTML  form  was  created  that  posted  the  survey  data  to  a  Microsoft  Access 
database. 


CAP  Survey  Form 

Code  Number  ERIMTEST 

1  Before  receiving  4ns  questionnaire,  did  you  know  any  of  the  informaaon  ta  the  box  below? 

Please  check  each  of  the  items  you  knew. 

n  R1  AEDA  inert  cemficatioos  now  require  two  signatures  from  either  qualified  DoD  or  > 

■■  contractor  personnel  ;  -j> 

r  R2  A  standard  memorandum  of  understanding  is  now  available  for  DRMO  use  with  all  j  tft 
of  their  AEDA  generators.  ;  p 

C!i  R3  When  turned  in  to  the  DRMO,  containers  placed  on  or  banded  to  pallets  must  ;  *7 

allow  for  visual  inspection  ofaB  containers.  i  ~ 

!  HR4  AEDA  Range  Residue  that  may  be  dangerous  to  public  health  and  safety  must  be  j  £-j 

rendered  safe  and  the  area  normally  containing  the  hazardous  material  must  be  j  ~  ■ 

opened  for  visual  inspection.  j  «*j! 

j  P  R5  Before  removal  of  AEDA  material,  qualified  individuals  wiE  bnef  die  purchaser  and  I  £ : 

DRMO  personnel  on  the  responses/actions  necessary  m  the  event  that  hve  or  j  Z: 

il  suspected  live  AEDA  is  discovered.  i  T ; 


2.  If  you  had  the  information  in  the  box  but  you  heard  it  differently,  please  write  the  facts  you  heard 
below.  Use  die  reference  numbers  R1  -  R5  from  above. 


IF 


If  you  checked  above  that  you  knew  any  of  the  information,  please  complete  the  questionnaire  by  ■ 
providing  the  information  requested  below.  Z 

If  you  did  not  check  any  of  the  items  (you  did  not  know  any  of  the  information  in  the  box),  you  have 
completed  the  questionnaire.  Please  submit  the  questionnaire  using  this  button.1  *1 


Figure  3  -  Using  a  Web-based  effectiveness  form  was  essential  at  DRMS 


4.2.5  Define  Sampling  Strategies 

Given  the  size  and  population  density  of  the  WSSCs,  the  team  could  readily  survey  all  WSSC  staff.  The  WSSC 
organization  is  flat,  limiting  the  importance  of  hierarchy.  However,  geography  is  an  issue.  Figure  4  illustrates  how 
dispersed  the  WSSC  staff  is  on  the  base.  The  distance  from  the  lower  right  comer  to  the  upper  right  comer  of  the 
diagram  is  over  0.5  mile.  This  distinction  was  considered  in  the  data  analysis. 
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Figure  4  -  Geography  an  issue  for  C-141  WSSC 


The  project  team  knew  from  the  DRMS  baselining  results  that  zone  and  DRMO  size  were  relevant.  Ideally,  our 
survey  could  reach  down  within  each  DRMO  to  specific  staff  members  at  different  levels  of  the  hierarchy.  The  team 
approached  DRMS  with  this  request.  It  turned  out,  however,  that  HQ  does  not  have  that  level  of  information.  While 
they  have  a  “Chiefs”  mailing  alias,  it  is  not  up  to  date.  In  addition,  for  some  DRMOs  the  list  contained  up  to  eight 
DMRO  staff.  It  took  over  two  weeks  to  get  an  up-to-date  list  of  all  of  the  DRMO  Chiefs.  (In  a  meeting  where  the 
team  developed  one  of  the  effectiveness  surveys,  an  HQ  manager  wished  he  had  the  email  addresses  of  the  DRMO 
staff  directly  affected  by  his  efforts.)  Timely  access  to  accurate  respondent  information  is  essential  to  making  a 
system  like  this  work.  Just  a  few  failures  will  poison  the  respondent  population  for  all  future  data  collection  efforts. 

4.2.6  Define  Analysis/Feedback  Methods 

After  some  discussion  in  our  meetings,  the  project  team  decided  to  try  a  variety  of  graphical  and  numerical  methods 
to  communicate  our  results  back  to  management. 

4.2.7  Refine  Forms  and  Methods 

The  change  agents  served  as  a  mechanism  for  reviewing  the  forms  and  methods  as  the  project  moved  forward. 
Concurrent  with  these  revisions,  the  team  worked  with  the  change  agents  to  plan  for  pilot  deployment.  This  project 
could  not  have  been  completed  without  the  active  support  of  our  contacts  at  each  organization. 

4.3  Pilot  Test 

While  the  team  could  readily  identify  these  events  with  our  research  sites,  the  transmission  of  these  messages 
occurred  according  to  the  schedules  of  the  staff  at  those  sites.  As  such,  the  project  schedule  was  at  the  mercy  of  their 
timing.  For  example,  while  DRMS  events  were  identified  in  December,  those  events  would  not  occur  until  March. 
(In  fact,  all  of  the  events  identified  in  December  did  not  occur,  causing  some  scrambling  to  find  new  candidates.)  In 
fairness  to  our  hosts,  in  most  cases  the  transmission  of  these  messages  was  subject  to  bureaucratic  processes  outside 
of  their  control.  It  is  brought  up  here  only  because  it  is  important  when  conducting  work  like  this  to  plan  for  such 
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delays.  Perhaps  if  an  effort  like  this  was  internal  to  a  single  organization,  more  control  and,  thus,  scheduling 
effectiveness  could  be  achieved. 

4.3.1  Test  with  3  Events 

The  plan  was  to  test  the  feedback  mechanism  with  at  least  three  (3)  communication  events.  The  project  team 
developed  two  versions  of  the  feedback  form  for  use  in  this  test: 

•  A  paper  version  that  was  substantially  similar  to  the  original  form  from  Davis  ( 1 953).  The  team  made  some 
minor  modifications.  Since  the  baselining  survey  results  showed  that  trust  in  the  host  organizations  was  low, 
two  questions  were  added  about  the  believability  of  the  messages.  The  team  also  added  some  response  options 
to  reflect  changes  in  technology  since  1953,  i.e.,  email,  FAX,  the  Internet,  and  the  World  Wide  Web. 

•  A  WWW  version  based  on  the  same  instrument.  Several  alternative  versions  were  developed  based  on  different 
ways  of  representing  the  selection  options  using  Web  form  functionality.  One  version  looked  the  same  as  the 
paper  version.  Another  used  pull-down  menus  for  each  multiple  choice  question.  After  some  internal  testing,  it 
was  agreed  that  while  the  pull  down  menus  made  the  form  shorter,  they  also  made  it  more  difficult  to  see  and 
compare  the  choice  options  for  some  questions.  (This  problem  could  be  exacerbated  for  respondents  with  small 
monitors.)  Based  on  fieldwork  data,  the  team  also  knew  that  many  people  like  to  print  things  such  as  this  for 
review.  When  printed,  the  pull-down  menu  version  did  not  list  any  of  the  menu  choices,  making  the  printout 
worthless.  For  these  reasons,  the  team  chose  the  version  that  mimicked  the  paper  version. 

4 3.1.1  Testing  at  WR-ALC 

The  C-141  WSSC  did  have  the  hierarchy  that  was  originally  envisioned  for  the  effectiveness  testing.  Based  on  the 
results  of  previous  research,  you  would  expect  that  there  would  be  some  attenuation  of  messages  when  they  have  to 
traverse  multiple  levels  of  organizational  hierarchy.  The  C-141  WSSC  was  relatively  flat.  However,  it  was 
geographically  dispersed  across  at  least  one  square  mile  of  WR-ALC.  Most  of  the  WSSC  staff  was  housed  in  a 
single  building.  The  Schedulers  (ALS  in  Figure  4)  sit  with  their  respective  airplanes  in  the  hangars  where  the  repair 
work  is  conducted. 

First  Test  at  WR-ALC.  The  communication  event  consisted  of  a  series  of  meetings  held  with  all  WSSC  staff 
members.  The  essential  facts  were: 

•  The  Site  Assistance  Visit  showed  that  the  implementation  of  the  AREP  process  still  has  a  long  way  to  go. 

•  The  main  green  area  identified  during  the  Site  Assistance  Visit  was  “AREP  Process  Implementation”. 

•  The  main  red  area  identified  during  the  Site  Assistance  Visit  was  “Conducting  Material  Forward  Looks”. 

•  The  upcoming  Operational  Readiness  Inspection  (ORI)  is  an  important  event  for  WR-ALC. 

The  WSSC  Chief  followed  up  the  meetings  with  an  email  to  all  staff  members. 

The  effectiveness  data  was  collected  five  to  seven  days  after  the  meetings.  In  developing  the  effectiveness 
instrument,  the  WSSC  chief  also  added  two  items  that  were  incorrect: 

•  The  areas  identified  as  “low”  during  the  Communications  Survey  have  all  been  resolved.  (Work  on  these  issues 
was  on  going.  Actually,  these  meetings  were  held  to  help  address  the  “low”  areas.) 

•  The  ORI  will  be  1-5  February.  (The  actual  dates  were  1-5  March.) 

The  project  team  collected  effectiveness  data  by  walking  through  the  work  areas  and  asking  WSSC  staff  to  complete 
the  survey.  The  same  interviewer  gathered  all  of  the  data.  Respondents  generally  took  one  to  five  minutes  to 
complete  the  survey. 

After  the  test,  the  survey  was  revised  in  minor  ways: 

•  The  instructions  were  changed  to  highlight  the  “first”  hearing  of  the  message  to  try  to  ensure  that  respondents 
would  consider  the  original  source  of  the  message. 

•  Instructions  were  added  to  questions  with  “fine-grained”  response  options,  i.e.,  questions  4  and  8,  to  try  to  get 
respondents  to  review  all  of  the  options  before  selecting  just  one.  For  example,  in  the  first  test  at  WR-ALC,  the 
message  was  passed  in  a  meeting.  Technically,  meetings  could  be  considered  face-to-face  communication  or 
talking  in  a  small  group.  This  was  in  addition  to  the  correct  answer,  attending  an  organized  meeting.  Hopefully, 
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if  the  respondents  review  all  of  the  response  options  before  selecting  they  can  determine  the  most  appropriate 
answer. 

A  problem  arose  in  the  data  analysis  with  respondents  saying  they  heard  a  falsehood  in  question  one  and  then 
correcting  it  in  question  two.  This  made  it  difficult  to  score  each  questionnaire  because  there  were  several  ways  to 
get  the  “right”  answer,  i.e.,  in  this  test  there  were  four  correct  statements  and  two  falsehoods.  A  respondent  could  be 
accurate  and  answer  in  four  different  ways: 

•  Say  they  heard  all  of  the  statements  and  then  correct  the  two  errors. 

•  Say  they  heard  the  four  truths  and  the  first  falsehood,  and  then  correct  that  falsehood. 

•  Say  they  heard  the  four  truths  and  the  second  falsehood,  and  then  correct  that  falsehood. 

•  Say  they  heard  just  the  four  truths. 

Recall  that  the  goal  was  for  the  survey  to  be  administered  and  scored  with  no  manual  intervention.  Therefore,  the 
administration  practices  and  instructions  have  to  be  changed  to  avoid  this  situation.  This  can  also  be  addressed  with 
instructions  to  the  manager  regarding  which  falsehoods  they  introduce.  In  this  case,  the  falsehood  sentences 
contained  true  subjects  (“areas  identified  as  low”  and  “ORI”)  but  specific  facts  about  those  subjects  were  false,  either 
status  (“all  resolved”)  or  the  date  (“1-5  February”).  If  the  process  does  continue  to  allow  for  falsehoods,  they  should 
be  more  along  the  lines  of  the  “popular  misconceptions”  that  are  completely  false. 

Since  geography  was  an  issue,  the  team  analyzed  the  data  by  location,  i.e.,  those  in  Building  125  (the  main  building 
in  Figure  4)  and  those  not  in  that  building.  To  draw  ready  comparisons,  the  team  wanted  to  the  report  to  allow  the 
readers  to  see  the  results  by  location  side-by-side.  It  was  decided  to  prepare  the  report  in  Microsoft  PowerPoint  so 
that  when  printed  in  handout  form,  six  slides  to  a  page,  the  results  from  each  location  appeared  side-by-side.  This 
report  is  included  in  Appendix  G. 

Second  Test  at  WR-ALC.  The  second  test  at  WR-ALC  focused  on  changes  to  a  logistics  support  contract  called 
the  Industrial  Prime  Vendor  (IPV).  This  contract  has  been  in  the  works  for  some  time.  This  communication  updated 
staff  members  on  the  program  status.  The  questionnaire  included  five  “facts”  about  facets  of  the  IPV 
implementation. 

The  information  was  conveyed  in  a  series  of  meetings  held  in  Building  125  at  WR-ALC  (one  small  group  in  an 
outlying  hangar  was  briefed  in  their  work  area).  Unfortunately,  the  briefings  were  delayed  until  the  day  the 
researcher  arrived  to  collect  data.  Consequently,  our  researcher  collected  effectiveness  data  right  after  each  briefing. 
Many  staff  members  (about  one  third  of  all  respondents)  showed  their  disapproval  of  this  technique  by  saying  they 
had  not  heard  any  of  the  factual  information  included  in  the  briefing  just  minutes  before. 

Third  Test  at  WR-ALC.  The  third  test  at  WR-ALC  was  conducted  simultaneously  with  the  second.  The  IPV 
briefing  also  contained  new  information  about  the  Forward  Support  Areas  (FSAs),  new  work  areas  being  constructed 
in  the  WR  hangars.  Again,  the  effectiveness  data  was  collected  right  after  the  briefing.  For  this  survey,  about  one 
third  of  the  respondents  (20  out  of  63)  chose  to  ignore  the  survey  questions. 

All  WR  testing  information  is  included  in  Appendix  G. 

4.3.1. 2  Testing  at  DRMS 

The  team  met  with  the  DRMS  contacts  in  December  and  identified  four  possible  communication  events: 

1.  HQ  staff  visited  every  DRMO  to  discuss  an  upcoming  “Commercial  Venture”,  one  of  the  Enterprise 
Management-related  changes.  These  meetings  had  already  occurred.  Since  it  would  take  some  time  to  initiate 
the  data  collection,  it  was  decided  to  drop  this  opportunity 

2.  A  communication  was  planned  about  needing  additional  data  from  some  DRMS  staff  to  support  Activity  Based 
Costing  (ABC).  The  information  would  pass  from  the  Zone  Managers  to  the  DRMOs  about  how  their  workload 
would  be  measured  against  hours  in  ABC. 
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3.  DRMS  was  working  with  the  National  Technical  Information  Service  (NTIS)  on  the  distribution  of  DRMS 
catalogs.  This  was  to  be  an  email  with  an  attached  spreadsheet.  It  would  describe  staff  responsibilities  and  what 
actions  might  be  required. 

4.  A  new  reimbursement  policy  on  scrap  was  being  developed  (Program  Budget  Decision  412).  It  would  most 
likely  be  an  email  with  a  hard  copy  letter  to  the  DRMO  Chief. 

These  events  were  all  to  occur  by  the  end  of  February.  Unfortunately,  the  events  selected  all  required  action  by 
outside  organizations.  Effectively  this  meant  that  they  were  all  out  of  the  control  of  the  DRMS  contacts.  It  also  did 
not  help  that  the  DRMS  point  of  contact  changed  three  times  between  December  and  the  first  test.  On  the  fourth 
POC,  the  team  decided  to  identify  new  events. 

Working  with  the  DRMS  contact,  the  team  identified  two  possible  events  and  worked  to  construct  the  respondent 
database.  The  events  were  policy  changes  that  were  communicated  from  HQ  to  the  Zone  Managers  and  on  to 
DRMO  Chiefs.  The  Chiefs  then  passed  on  the  information  to  their  first  line  supervisors  who,  in  turn,  communicated 
with  some  of  their  staff.  During  baselining,  the  team  determined  that  almost  all  of  these  people  have  individual  email 
addresses  and  access  to  the  WWW.  Ideally,  the  team  wanted  to  track  the  progress  of  these  messages  through  this 
network.  Unfortunately,  this  level  of  respondent  information  was  not  available. 

As  a  fall  back  position,  it  was  decided  to  reach  the  DRMO  staff  through  their  Chiefs.  First,  the  DRMO  Chiefs  were 
listed  for  each  Zone  Manager.  The  DRMO  Chiefs  were  then  used  to  solicit  responses  from  their  first  lines  and  other 
staff.  To  help  increase  the  response  rate  from  the  chiefs  and  to  increase  their  compliance  with  our  request,  the 
request  was  sent  out  as  an  email  from  the  DRMS  Commander.  Even  this  level  of  information  was  difficult  to  obtain. 

All  of  the  DRMS  surveys  and  results  are  included  in  Appendix  E. 

First  Test  at  DRMS.  In  early  April,  a  change  in  the  Bidder  registration  process  was  communicated  from  DRMS 
HQ  to  the  Zone  Manager  and  DRMO  Chiefs.  Since  this  policy  affected  bidding.  Distribution  Branch  Chiefs,  Sales 
Contracting  Officers,  Cashiers,  and  Marketing  Technicians  would  need  to  know  about  the  change.  The  original 
message  requested  that  all  affected  parties  review  the  details  of  the  change  available  on  the  DRMS  Internal  World 
Wide  Web  site.  (While  the  team  was  told  this  was  a  “change”,  it  was  more  of  a  clarification  of  desired  best 
practice.) 

Respondents  for  this  survey  were  solicited  using  emails  sent  to  the  Zone  Managers  and  DRMO  Chiefs.  The  Zone 
Managers  were  provided  a  URL  to  a  password-protected  WWW  site.  They  were  provided  a  user  name,  password, 
and  code  number  to  identify  them  by  Zone.  Chief  were  sent  a  similar  email,  requesting  that  they  respond  and  to 
forward  a  similar  request  to  the  appropriate  staff  members.  This  indirect  contact  was  necessary  since  HQ  could  not 
provide  direct  email  contact  with  the  specific  staff  categories.  For  this  same  reason,  the  team  can  only  estimate  the 
possible  number  of  respondents.  Based  on  12  Zone  Managers  and  83  DRMOs,  it  was  estimated  that  there  could  be 
approximately  400  respondents. 

1 14  responses  were  received,  a  29%  response  rate.  This  is  reasonable  for  a  survey  of  this  type.  Response  rates  by 
zone  vary  widely,  as  shown  in  Figure  5.  (The  12  Zones  were  arbitrarily  numbered  from  1  to  12.) 

Respondents  did  not  score  well  on  their  knowledge  of  this  “new”  process.  New  is  in  quotes  because  during  survey 
construction  it  was  portrayed  to  the  project  team  as  such.  As  the  open-ended  responses  made  clear,  this  process  was 
not  new,  making  the  questions  confusing.  Further  discussion  with  DRMS  staff  revealed  that  this  communication  was 
intended  to  clarify  best  practice  to  the  field. 

Zone  1  Zone  2  Zone  3  Zone  4  Zone  5  Zone  6  Zone  7  Zone  8  Zone  9  Zone  1 0  Zone  1 1  Zone  1 2 

Rate  46  36  18  24  14  10  32  7  36  29  50  35 

Figure  5  -  Response  rates  by  Zone  for  DRMS  Survey  1 
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While  there  may  have  been  some  confusion  as  to  whether  this  indeed  was  a  new  process  or  not,  many  respondents 
were  not  clear  about  what  the  desired  best  practice  is  for  bidder  registration.  The  document  issued  from 
Headquarters  contained  some  general  information  about  what  should  be  done.  However,  the  desired  behaviors 
required  some  advanced  skills,  such  as  complex  Boolean  searching  to  find  any  duplicate  names  in  the  databases  and 
the  simultaneous  use  of  two  different  applications  programs,  DNSP  and  CORALS.  The  results  of  the 
communications  baselining  survey  conducted  as  part  of  this  study  illustrated  that  field  staff  have  had  little  formal 
training  on  many  of  the  computer-based  tools  they  must  use  everyday.  It  was  recommended  to  DRMS  that  future 
process  clarifications  that  require  the  mastery  of  advanced  computing  skills  be  accompanied  by  more  detailed 
process  descriptions,  including  relevant  examples  using  graphics  and  screen  captures  from  the  required  applications 
programs. 


Second  Test  at  DRMS.  In  early  April,  the  publication  of  revised  guidance  for  the  processing  of  Ammunition, 
Explosives,  and  Other  Dangerous  Articles  (AEDA)  and  Range  Residue  communicated  from  DRMS  HQ  to  the 
DRMS  Vice  Presidents,  Zone  Managers,  and  DRMO  Chiefs.  This  policy  change  should  have  been  communicated 
down  the  following  hierarchy: 

The  DRMO  Chief  to: 

•  The  Property  Management  Branch  to  the  DEMIL  Coordinator; 

•  The  Distribution  Branch  Chief  to  the  Marketing  Specialist(s);  and 

•  The  Environmental  Branch  Chief 

The  original  email  message  requested  that  all  affected  parties  review  the  details  of  the  change  available  on  both  the 
DEMIL  page  of  the  DRMS  Internal  World  Wide  Web  site  and  on  the  DoD  Demilitarization  and  Trade  Security 
Controls  Web  site. 

91  responses  were  received,  an  18%  response  rate.  This  was  still  reasonable  for  a  survey  of  this  type,  but  was  much 
lower  than  for  our  first  survey.  There  are  several  possible  explanations  for  this  difference.  The  first  survey  was  sent 
out  from  the  DRMS  Commanders  email  address,  die  second  from  a  HQ  staffer.  The  second  solicitation  was  sent  out 
soon  after  the  first  and  may  not  have  been  distinguished  as  a  separate  survey.  Finally,  some  respondents  may  have 
simply  declined  to  be  surveyed  again. 

This  change  was  disseminated  much  faster  than  in  the  previous  survey,  with  a  higher  comprehension  rate.  This  was 
despite  the  fact  that  these  changes  were  buried  in  revised  DoD  standards.  The  DRMO  staff  should  be  commended 
on  their  ability  to  decipher  the  changes.  If  the  specific  changes  were  highlighted  as  part  of  the  communication,  it 
might  have  been  easier  on  all  of  those  who  have  to  learn  about  them.  From  the  open-ended  responses,  it  was  clear 
that  many  people  knew  that  these  changes  were  in  the  works  for  some  time. 

One  point  made  by  a  respondent  is  worth  noting,  however.  In  the  original  message,  the  respondents  were  told  to  go 
to  the  DRMS  DEMIL  page  to  find  the  necessary  information.  It  took  four  screens  of  information  to  get  to  that  point. 
One  ready  benefit  of  the  WWW  is  that  it  is  possible  to  directly  address  any  resource  (i.e.,  URL)  on  the  Web.  It  was 
recommended  to  DRMS  that  future  communications  make  it  easier  on  receivers  by  providing  the  direct  URLs. 

Tests  1  and  2  occurred  in  rapid  succession,  with  no  real  chance  to  make  changes  in  administration  practices.  During 
data  analysis  the  team  noticed  that  the  procedure  to  get  to  individual  DRMO  staff  was  not  working  as  intended.  The 
data  apparently  included  multiple  responses  from  each  DRMO  Chief.  (This  occurred  for  12  out  of  83  DRMOs  on 
the  first  survey  and  1 1  in  the  second.)  However,  these  responses  had  different  answers  and  came  from  different 
Internet  Protocol  (IP)  addresses  at  different  times.  The  most  likely  explanation  is  that  Chiefs  passed  the  whole 
message  to  them  to  their  subordinates,  not  just  the  part  that  the  team  had  requested  they  pass  on.  To  address  this,  a 
different  approach  was  employed  on  the  third  survey.  The  Chiefs  were  sent  two  messages: 

1 .  One  message  requesting  their  participation  and  provided  the  needed  code.  They  were  told  they  would  get  a 
second  message  that  they  could  pass  on  in  its  entirety  to  their  staff. 

2.  A  second  message  just  for  their  staff. 
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After  this  change,  the  problem  recurred  8  times  on  survey  3. 

Third  Test  at  DRMS.  In  late  April,  DRMS  HQ  published  a  memorandum  to  the  National  Region  Manager, 
National  Zone  Managers,  and  the  DRMO  Chiefs  about  the  demanufacturing  contracts  for  electrical  and  electronic 
scrap.  This  policy  change  should  have  been  communicated  down  the  following  hierarchy: 

The  DRMO  Chief  to: 

•  The  Property  Management  Branch  to  the  Demil  Coordinator,  Receiving  Coordinator(s)/Schedulers,  and 
Receivers. 

•  The  Distribution  Branch  Chief  to  the  Marketing  Specialist(s);  and 

•  The  Environmental  Branch  Chief 

Based  on  12  Zone  Managers  and  83  DRMOs,  it  was  estimated  that  there  could  be  approximately  510  respondents. 
92  responses  were  received,  an  18%  response  rate. 

For  surveys  2  and  3,  response  rates  were  reported  by  Zone,  responses  by  Chiefs  per  Zone,  and  the  response  rate  for 
Zone  Managers  overall.  For  example,  Figure  6  includes  this  information  for  Survey  3. 
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Figure  6  -  Response  rates  for  DRMS  Survey  3 


For  this  message,  respondents  scored  quite  well  overall  but  18%  had  never  heard  about  the  change,  a  cause  for 
concern  for  management. 

At  the  request  of  DRMS,  an  open-ended  item  was  added  to  this  last  survey,  where  respondents  could  sound  off  about 
our  surveys,  the  message  in  question,  or  any  other  topic.  DRMS  people  took  great  advantage  of  this  opportunity, 
with  16  respondents  providing  some  useful  information  that  was  passed  on  to  DRMS  in  its  entirety  (after  removing 
any  part  of  the  response  that  could  help  identify  its  source).  Since  this  was  not  part  of  the  original  project  plan  and 
was  added  only  for  the  customer,  the  team  did  not  expend  any  effort  to  code  the  open-ended  responses. 


4.3.2  Assess  Results 

The  reports  included  in  the  Appendices  were  provided  to  each  organization.  Because  many  of  these  tests  were 
delayed  until  the  last  few  weeks  of  the  project,  there  was  no  time  or  resources  to  visit  each  organization  in  person. 

4.3.3  Communicate  Results  to  Each  Organizational  Unit 

These  summaries  were  reviewed  with  the  organizational  units,  providing  valuable  input  to  the  revisions  to  the  forms 
and  methods. 


4.3.4  Revise  Forms  and  Methods 

As  described  in  the  previous  sections,  a  number  of  changes  were  made  to  the  data  collection  instrument  between 
tests.  These  changes  are  described  in  the  prose  above  and  can  be  seen  in  the  different  forms  included  in  the 
Appendices. 
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5.0  DISCUSSION 

Overall,  the  project  produced  some  positive  results,  albeit  over  a  much  longer  period  than  originally  planned.  The 
initial  schedule  covered  8  months.  In  the  end,  the  project  lasted  for  15.  The  team  tried  to  make  our  interactions  with 
the  research  sites  as  efficient  as  possible.  The  visits  were  planned  carefully  before  arriving  to  make  sure  as  much  of 
the  needed  information  as  possible  was  obtained.  One  important  lesson  was  that  outside  consultants  (essentially  the 
role  played  by  the  project  team)  can  have  greater  success  in  a  shorter  period  if  they  plan  to  have  their  collaborators 
do  most  of  what  they  need  during  a  planned  meeting.  When  the  team  worked  this  way,  the  support  needed  was 
acquired  most  of  the  time.  When  the  collaborators  were  left  to  their  own  devices  their  real  jobs  took  precedence  (as 
one  would  expect). 

Some  of  this  delay  was  unavoidable.  Trying  to  do  action  research  can  be  difficult  enough.  When  the  host 
organization  is  in  a  period  of  great  change,  the  challenge  is  much  greater.  This  is  the  downside  of  being  “outsiders”. 
It  is  harder  to  get  support  for  tasks  that  are  off  the  critical  path  of  your  subjects.  On  the  other  hand,  being  an  outsider 
in  this  situation  had  some  big  advantages.  The  response  rate  to  the  DRMS  baselining  survey  was  approximately 
60%,  with  few  callbacks  or  extra  effort  taken  to  pump  up  the  numbers.  This  cooperation  continued  with  all  of  the 
candid  responses  to  the  open-ended  questions  in  the  effectiveness  tests.  The  team  does  not  believe  that  an  “inside” 
organization  could  have  gotten  the  same  result.  The  bottom  line  is  that  when  a  plan  is  created  to  conduct  similar 
work,  make  sure  that  your  timelines  account  for  these  natural,  unavoidable  delays. 

The  project  was  based  on  an  important  assumption,  i.e.,  the  team  could  readily  use  data  collection  instruments  tested 
by  other  researchers  for  our  purposes.  In  practice,  the  project’s  experience  with  this  assumption  was  mixed. 

The  baselining  survey  produced  some  important  results.  At  WR-ALC,  the  project  team  knew  a  lot  about  the  culture 
and  issues  facing  the  organization  from  observation  and  anecdotal  sources.  These  beliefs  were  strongly  supported  by 
the  empirical  results.  The  DRMS  results  gave  us  valuable  insights  into  the  nature  of  the  organization,  highlighting 
the  split  between  the  HQ  and  DRMOs.  As  described  in  the  previous  sections,  the  survey  included  several  sections, 
two  of  which  came  from  existing  surveys.  The  Glaser  et  al  organizational  culture  survey  provided  a  quick  diagnosis 
of  important  organizational  issues,  such  as  meetings,  teamwork,  and  supervision.  The  face  validity  of  these  results  at 
WR-ALC  caused  the  C-141  WSSC  Chief  to  take  immediate  action  on  some  issues.  The  Roberts  and  O’Reilly 
instrument  allows  you  to  dig  deeper  into  many  issues,  such  as  sources  and  uses  of  information.  For  example,  the 
survey  results  showed  differences  in  communication  sources  between  HQ  DRMS  and  the  DRMOs.  At  HQ,  people 
get  most  of  their  information  from  peers,  while  in  the  field  communication  is  much  more  hierarchical. 

The  sections  that  were  added,  on  computing  issues  and  facts  about  the  current  organizational  change,  worked  well. 

At  DRMS,  there  was  some  confusion  about  our  directions  on  the  computing  issues  section.  While  the  printed 
instructions  said  that  only  DRMO  Chiefs  were  to  complete  those  questions,  many  respondents  provided  data.  To 
correct  this  the  team  did  callbacks  with  each  Chief  to  collect  the  data  directly.  The  fact  gathering  process  resulted  in 
some  useful  data  that  helped  the  team  understand  the  target  organizations  better.  Turning  this  data  into  questionnaire 
items  was  one  major  source  of  delay.  The  DRMS  baselining  survey  lagged  behind  the  WR-ALC  survey  by  almost 
two  months  because  the  team  had  to  wait  for  input.  Once  the  items  were  created,  the  respondents  at  DRMS  and  the 
F-15  WSSC  did  not  demonstrate  adequate  knowledge  of  these  facts.  The  F-15  WSSC  staff  scored  quite  well.  The 
baselining  results  showed  that  the  F-15  group  scored  higher  on  the  morale  and  trust  questions  (our  fieldwork 
illustrated  this  qualitatively).  You  would  expect  that  people  with  higher  morale  and  more  trust  might  be  more 
motivated  to  participate  in  surveys  (and  probably  would  be  more  motivated  in  their  jobs).  This  can  explain  some  of 
the  difference.  Another  possible  explanation  is  that  the  surveys  did  not  ask  them  the  right  facts.  However,  the  team 
used  most  of  the  same  information  sources  that  staff  members  would  use  to  get  this  information  (except  the 
grapevine,  which  the  team  had  no  systematic  way  to  tap).  These  facts  were  also  checked  with  many  of  the  managers 
who  are  responsible  for  communicating  the  on-going  changes  with  the  employees.  The  bottom  line  is  that  the 
respondents  did  not  know  this  information.  It  is  up  to  management  to  determine  an  appropriate  course  of  action. 

Our  experience  with  the  ECCO  instrument  was  not  as  positive.  Some  changes  were  mundane,  such  as  adding 
response  options  to  reflect  communication  technologies  such  as  FAXs  and  the  Internet.  However,  the  respondents 
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quickly  pointed  out  some  other  necessary  changes  that  were  subtler.  For  example,  the  survey  only  allowed 
respondents  to  “receive”  information  from  one  of  four  people:  supervisors  above  their  immediate  supervisor,  their 
supervisor,  their  peers,  or  their  subordinates.  With  all  of  the  communication  options  available,  this  question  needed 
the  “None  of  the  Above”  that  was  added  for  the  later  DRMS  test.  The  biggest  problem  at  both  sites  was  in  the 
“talking/sound”  methods  of  receiving  information.  People  had  a  hard  time  distinguishing  between  “face-to-face”, 
“small  groups”,  and  “organized  meetings”.  At  first,  the  team  thought  that  people  were  not  reading  all  of  the  options 
before  deciding.  However,  when  the  instructions  were  changed  on  that  question  to  explicitly  ask  the  respondents  to 
review  all  options  before  selecting  the  best,  the  problem  still  occurred.  (One  could  argue  that  the  people  were  not 
reading  the  instructions  either,  but  there  is  only  so  much  that  a  survey  administrator  can  do.) 

The  team  unintentionally  caused  some  administration  problems.  The  first  WR-ALC  effectiveness  survey  included 
four  “true”  statements  and  two  “false”  statements,  as  described  in  the  section  on  that  test.  This  caused  problems  in 
the  analysis  that  would  be  difficult  to  automate.  One  way  around  this  in  future  applications  is  to  use  true/false 
questions. 

There  was  some  discussion  among  our  team  about  the  large  difference  in  open-ended  comments  between 
participating  organizations.  The  DRMS  respondents  used  the  open-ended  items  as  an  opportunity  to  vent  their 
frustration.  The  WR-ALC  respondents  said  almost  nothing.  As  researchers,  the  open-ended  responses  helped  us 
interpret  the  quantitative  data.  The  team  hopes  that  the  DRMS  managers  found  the  information  as  valuable. 

5. 1  High  Level  Software  Requirements 

The  project  team  was  fortunate  to  be  able  to  conduct  some  surveys  using  the  WWW.  As  originally  planned,  this 
pilot  included  only  paper  surveys.  Some  internal  staff  changes  and  the  addition  of  DRMS  as  a  research  site  made  the 
use  of  the  WWW  a  necessity.  ERIM  hosted  the  site  on  its  secure  Web  server.  (Based  on  ERIMs  experience  with 
projects  like  this,  users  feel  better  if  their  interactions  occur  in  a  secure  environment,  even  if  the  data  is  not 
particularly  sensitive  or  valuable.  The  project  team  recommends  that  any  system  like  this  be  housed  on  a  secure 
server.) 

A  HQ  DRMS  staff  member  sent  respondents  a  custom  email.  (These  messages  were  created  using  Microsoft  Office 
applications.  The  process  could  readily  be  automated  using  Visual  Basic.)  The  message  included  a  URL  for  a 
welcome  page.  Once  they  arrived  at  this  page,  they  entered  a  username  and  password  provided  in  the  email.  Once 
logged  in,  they  entered  a  code  number  from  the  email  to  get  to  the  survey. 

The  survey  form  was  supported  by  some  error  checking  code.  This  greatly  reduced  erroneous  answers.  After 
completing  the  survey  and  clicking  on  the  “Submit”  button,  the  data  was  posted  to  a  Microsoft  Access  database.  The 
team  believes  that  these  tools  could  be  readily  integrated  to  develop  this  system.  A  diagram  of  the  system  is  shown 
in  Figure  7.  The  “DB  Schema”  interface  is  part  of  the  administration  subsystem.  As  seen  in  this  project,  having  up- 
to-date  respondent  information  is  one  of  the  most  critical  parts  of  making  this  system  work.  A  second  interface,  the 
“Enter  Data”  screen,  will  be  required  for  managers  to  create  the  effectiveness  instruments  and  describe  the  desired 
respondents.  Managers  will  need  Web  access  to  create  these  forms  and  to  review  the  results.  An  email  address  will 
be  required  so  that  they  can  be  notified  when  the  analysis  is  completed.  Microsoft  Excel  was  used  as  the  “data 
processing  module”  for  the  pilot.  A  production  system  might  be  better  served  by  something  more  powerful,  although 
the  HTML  creation  capabilities  of  Microsoft  tools  make  them  attractive.  Finally,  once  the  analysis  is  complete, 
managers  (and  respondents)  would  be  directed  to  review  the  results  on  the  Web. 

One  development  lesson  learned  on  the  RAPTR  project  needs  to  be  repeated  here.  If  possible,  try  to  use  plain 
“vanilla”  WWW  techniques.  On  RAPTR  that  project  team  got  into  trouble  when  the  software  development 
contractor  put  in  features  that  required  a  specific  version  of  a  specific  browser  for  the  system  to  work  as  intended. 
Since  any  CAP  system  may  be  deployed  in  organizations  that  do  not  have  a  homogeneous  computing  environment, 
taking  this  approach  can  avoid  some  problems.  At  the  very  least,  the  development  team  should  test  the  software  on 
several  different  revisions  of  the  possible  browser  software  that  is  resident  on  the  possible  hardware  that  respondents 
may  have  installed. 
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» Web  page  creation 


*  Summary  statistics 

•  “Best”  recommendations 


Figure  7  -  Structure  of  automated  CAP  System 


As  far  as  siting  this  system,  the  CAP  team  firmly  believes  that  this  project  got  better  data  because  it  was  outside  the 
host  organizations.  Having  an  outside  organization  provide  this  function  is  something  to  consider.  At  the  very  least, 
the  function  should  be  housed  in  a  trusted  organizational  unit.  Perhaps  a  human  resource  organization  makes  sense. 
If  your  organization  has  an  ombudsman,  this  may  be  an  option.  Trust  is  crucial  to  making  this  work  for  your 
organization. 


6.0  CONCLUSION 

Without  question,  effective  communication  is  a  critical  characteristic  of  today’s  world-class  organizations.  The  CAP 
effort  successfully  developed  a  procedure  for  near-real  time  assessment  of  the  effectiveness  of  organizational 
communications.  Such  a  procedure  can  have  tremendous  payoff  to  an  organization  by  identifying  communication 
problems  and  their  sources  in  a  timely  manner,  so  that  remedial  action  can  be  taken  by  management  to  resolve  them. 
The  procedure  developed  in  this  effort  is  largely  manual;  however,  the  foundation  has  been  laid  for  a  software  tool  to 
automate  the  procedure  in  follow-on  efforts. 

The  next  step  to  is  construct  an  automated  system  based  on  the  results  and  requirements  laid  out  in  this  report.  Once 
the  system  is  in  place,  the  goal  is  to  use  it  to  test  a  large  number  of  messages  of  different  types  over  an  extended 
period.  As  in  CAP,  the  system  would  be  used  to  provide  quantitative  feedback  on  the  extent  and  effectiveness  of 
specific  communication  events.  In  parallel,  researchers  would  conduct  content  analyses  on  these  events  that  would 
serve  two  purposes.  The  researchers  would  provide  qualitative  feedback  on  each  event,  just  as  in  CAP.  The  second 
purpose  has  longer-range  implications.  The  researchers  would  development  message  types  categories,  e.g., 
informational,  directive,  etc.  The  communication  literature  has  some  results  on  matching  message  content  with 
media  type.  In  the  second  phase  of  CAP,  the  researchers  would  attempt  to  add  to  this  knowledge  base  by  correlating 
message  types  and  effectiveness  results.  The  goal  is  to  provide  more  directed  feedback  to  management  and  to 


17 


provide  the  data  to  construct  a  management  communication  learning  system.  This  system,  which  could  be  a  third 
CAP  phase,  would  guide  managers  who  are  planning  communication  events  to  select  the  appropriate  means  given 
their  communication  goals.  Specific  examples  of  successful  (and  unsuccessful)  communications  could  be  drawn 
from  the  second  CAP  phase  to  provide  additional  guidance.  With  these  tools  and  this  approach,  managers  could 
maximize  the  likelihood  of  communication  success  before  they  send  out  messages  and  then  get  timely  feedback  for 
that  message.  Closing  the  feedback  loop  in  this  manner  is  essential  to  improving  management  communications. 
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Appendix  A. 

Acronyms 

ABC 

Activity  based  costing 

ACM 

Association  for  Computing  Machinery 

AREP 

Aircraft  Repair  Enhancement  Program 

BPR 

business  process  reengineering 

CAP 

Communications  Assessment  Pilot 

CDRL 

Contract  Data  Requirements  List 

CEC 

Center  for  Electronic  Commerce 

DAC 

Days  After  Contract 

DoD 

Department  of  Defense 

DRMO 

Defense  Reutilization  and  Marketing  Office 

DRMS 

Defense  Reutilization  and  Marketing  Service 

ERIM 

Environmental  Institute  of  Michigan 

FSA 

Forward  Support  Areas 

HQ 

Headquarters 

IPV 

Industrial  Prime  Vendor 

ITI 

Industrial  Technology  Institute 

LAN 

local  area  network 

MAC 

Months  After  Contract 

NR 

Non  response 

ORI 

Operational  Readiness  Inspection 

PD 

Product  Directorate 

POC 

Point  of  contact 

RAPTR 

Readiness  Assessment  and  Planning  Tool  Research 

RE 

WR-ALC  Reengineering  Directorate 

SAV 

Site  Assistance  Visit 

SOW 

Statement  of  Work 

WR-ALC 

Wamer-Robins  Air  Logistics  Center 

WSSC 

Weapon  System  Support  Center 

WWW 

World  Wide  Web 

\ 
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Appendix  B.  Objectives  for  Site  Visits 

1 .  We  must  determine  what  receivers  should  currently  know  about  specific  change  efforts  through  interviews  of 
communicators  and  content  analysis  of  recent  communication  events. 

•  Review  “facts”  lists  -  revise  as  required;  probably  best  accomplished  in  a  group  process  (get  more  views, 
better  consensus) 

•  Prioritize,  cull  most  important 

2.  We  also  must  determine  the  anticipated  frequency  and  nature  of  communication  events  and  how  often 
monitoring  events  may  be  required.  Are  there  regularly  scheduled  briefings?  Do  status  reports  or  other 
documents  have  a  standard  format  and  come  out  on  a  regular  schedule?  This  will  help  scope  out  just  how  many 
events  may  have  to  tracked  by  the  proposed  software  system. 

•  Need  more  understanding  of  events 

•  E.g.,  DRMS  example  recent  training  guidelines  on  Intranet 

3.  Determine  the  desired  sampling  characteristics,  i.e.,  what  criteria  would  managers  of  the  organizational  units 
like  to  use  to  select  respondents  and  view  summary  statistics  about  their  communication  effectiveness?  While 
there  is  some  flexibility  to  change  sampling  criteria  and  monitoring  frequency  during  the  manual  pilot,  we  need 
to  be  rigorous  to  make  sure  we  specify  all  possible  sampling  characteristics  for  the  proposed  software  system. 

•  Define  sampling  criteria 

•  Have  natural  break  between  HQ  and  field 

4.  Need  to  discuss  how  results  might  be  presented 

Have  ECCO  instrument  as  baseline;  we  can  discuss  it  and  what  additions,  e.g.,  demographics,  that  would  have  to 
be  added  to  do  some  categorization  of  the  results. 

5 .  Review  evolving  materials  with  management 

•  Baselining 

•  Demographics 

•  Culture 

•  Communications  Climate 

•  Infrastructure 

•  Facts 

•  Effectiveness 

•  ECCO 

•  Cover  letter  for  each  organization 

•  DRMS  -  should  we  put  up  on  Intranet 

•  Project  description 

•  Cover  letter 

•  Survey  results 

6.  Sketch  out  schedule  for  baselining  and  event  analysis 
Key  Participants: 

•  DRMS  -  Marketplace  Group  -  Woosley,  Mank,  Beam;  also  see  Schaberg,  Simpson,  Allred 

•  F-15  -  Tony  Delgiomo,  Karen  Freyermuth;  C-141  -  Abe  Banks 
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Appendix  C.  Literature  Review 

This  appendix  includes  the  results  of  the  literature  review  conducted  by  Dr.  Julia  Gluesing  of  Cultural  Connections. 
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Cultural  Connections,  Inc 


P.  O.  Box  4418 
Troy,  Ml  48099-4418 
phone  248.879.9619 
fax  248.879.9081 
e-mail  info@teamcci.com 
web  http://www.teamcci.com 


Memo 


To:  Stan  Przybylinski 

From:  Julia  Gluesing 

CC: 

Date:  May  11, 1998 

Re:  CAP  Proj  ect  -  Communication  Assessment  Tools 


I  have  conducted  a  good  first  pass  through  the  communication  literature,  and  related  management  and  organizational 

literature,  to  locate  tested  communication  effectiveness  assessment  tools  that  would  fit  the  objectives  of  the  CAP  project. 

Preliminary  Results: 

■  The  ECCO  survey  seems  to  be  the  best  fit  for  evaluation  of  a  specific  target  message,  tracing  its  flow  and  the  recipients 
evaluation  of  the  message  content.  This  instrument  is  also  one  of  the  most  objectives  measures  I  have  found,  since  it 
does  not  rely  on  opinions  or  attitudes,  but  on  the  reporting  of  communication  events. 

■  Most  other  assessment  instruments  look  not  at  specific  messages  but  at  perceptions  about  the  quality  of  communication 
in  general  and  about  the  factors  that  influence  the  quality  of  communication.  Several  instruments  fall  into  this  category. 
I  have  attached  a  summary  of  my  findings  to  this  memo. 

■  Message  effectiveness  seems  to  have  gone  by  the  wayside  as  a  research  topic  in  recent  years,  having  been  replaced  by 
broader  issues  related  to  relationships  and  systems.  The  more  recent  research  appears  to  be  focused  more  on 
communication  systems,  and  larger  issues  and  on  the  effects  of  interpersonal  and  cultural/climate  characteristics  as 
variables  of  influence  on  internal  communication  effectiveness.  The  literature  also  indicates  a  significant  increase  in 
electronic  and  computer-mediated  communication  as  a  focus  of  study  and  in  network  studies  (these  studies  were  out  of 
fashion  in  the  80s).  The  issues  being  researched  and  measured  are  related  to  system/network  processes  (both  formal 
and  informal),  and  on  the  conditional  context  in  which  communication  takes  place. 
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Recommendations  and  Next  Steps: 

■  Review  search  results  and  recommendations  with  you. 

■  Continue  a  more  in-depth  investigation,  if  necessary/desired. 

■  Use  the  ECCO  instrument  as  the  primary  data  collection  tool,  and  create/use  other  indices  to  supplement  this 
instrument.  I  have  a  few  suggestions  about  how  to  do  this  in  the  attached  pages. 

Let  me  know  how  you  wish  to  proceed.  Thanks. 
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Summary  of  initial  literature  search/review: 

•  I  did  finally  locate  the  reference  to  the  ECCO  instrument  that  we  have  all  been  referring  to: 

Goldhaber,  G.M.,  Dennis,  H.  S.  Ill,  Richetto,  G.  M.,  &  Wiio,  O.  A.  (1986).  Information  strategies:  New 
pathways  to  management  productivity .  Norwood,  NJ:  Ablex.  Pp  238-240. 


This  book  also  has  some  other  useful  instruments. 


1.  The  ICA  Audit  and  the  OCD  Audit  that  you  are  familiar  with  from  the  Communication  Research  Measures 
book. 

2.  There  is  also  an  instrument  on  Assessing  Communication  Contingencies,  pp.  71  -  72,  which  might  be 
useful.  I  have  faxed  you  these  pages  from  the  book. 

3.  On  pages  194-202,  there  is  an  instrument  that  assesses  the  management  communications  system.  We  might 
find  this  instrument  useful  to  assess  the  communication  system  as  a  whole  -  it  could  be  a  periodic 
assessment,  and  could  be  worked  into  a  web-based  assessment  system. 

4.  Another  related  instrument  for  assessing  the  conditional  context  in  which  communication  takes  place  is  the 
Dennis  Communication  Climate  Inventory.  I  have  faxed  you  a  copy  of  this  as  well.  It  is  on  pages  234  - 
237 

Overall,  this  book  is  very  good  at  providing  the  background  and  justification  for  several  of  the  communication 
assessment  methods  and  tools.  You  can  try  to  get  a  copy  of  the  book,  or  I  can  share  the  copy  I  have  with  you. 

•  The  following  book  of  measurement  had  some  indices  that  could  be  coupled  with  the  ECCO  instrument  to 
provide  a  more  complete  assessment  of  communication  effectiveness: 

Price,  James  L.  &  Mueller,  C.  W.  (1986).  Handbook  of  organizational  measurement.  Marshfield,  MA: 
Pitman. 

I  have  faxed  you  the  relevant  pages  from  the  Chapter  on  Communication  -  pp  83  85  and  90  -  96. 

1.  Pages  83  -  85,  while  they  do  not  contain  a  ready-made  instrument,  suggested  that  there  are  four 
dimensions  of  communication  that  should  be  assessed:  formal — informal,  vertical — horizontal, 
personal — impersonal,  and  instrumental — expressive. 

2.  Pages  90  -  94  present  the  O’Reilly  and  Roberts  instrument  to  measure  source  credibility  and 
communication  behavior.  This  is  probably  the  instrument  most  closely  related  to  the  objectives  in  the 
CAP  project. 

3.  Pages  94  -  96  discuss  additional  suggestions  for  measurement  of  work  units.  The  authors  suggest  the 
creation  of  an  index  that  measures  six  features  of  the  work  unit  having  to  do  with  meetings  and  their 
characteristics.  This  could  also  be  useful  if  we  want  to  construct  some  added  instruments  to  go  along 
with  the  ECCO  instrument. 

•  There  is  also  a  chapter  in  one  of  the  older  communication  yearbooks  that  presents  a  review  of  the  literature  on 
the  criteria  for  organizational  communication  effectiveness.  The  chapter  lays  out  the  criteria,  but  does  not 
provide  any  assessment  instruments.  The  criteria  provide  good  justification  for  creating  additional  questions,  or 
a  new  index  that  we  could  couple  with  the  ECCO  instrument.  I  have  faxed  you  a  copy  of  this  article. 

Farace,  R.  V.,  Taylor,  J:  A.,  &  Stewart,  J.  P.  (1978).  Criteria  for  evaluation  of  organizational  communication 
effectiveness:  Review  and  synthesis.  In  B.  Ruben  (Ed.)  Communication  Yearbook  2.  Pp.  271-292. 
New  Brunswick,  NJ:  Transaction  Books. 

•  Two  references  appear  in  the  book  by  Cynthia  Stohl  (1995).  Organizational  communication:  Connectedness  in 
action.  Newbury  Park,  CA:  Sage.  These  references  could  be  useful  if  we  want  to  assess  the  quality  of  the 


25 


messages  in  terms  of  intent  versus  received  meaning,  or  if  we  want  to  see  how  many  communication  steps  it 
takes  to  get  from  the  source  to  the  receiver.  My  sense  is  from  the  study  objectives,  that  this  information  might 
be  incorporated  as  part  of  the  feedback  data  reported  to  message  senders. 

1.  Stohl,  C.,  &  Redding,  W.  (1987).  Messages  and  message  exchange  processes.  In  F.  Jablin,  L.  Putnam,  K. 
Roberts,  &  L.  Porter  (Eds.),  Handbook  of  organizational  communication:  An  interdisciplinary  approach 
(pp.  451-502).  Newbury  Park,  CA:  Sage. 

According  to  Stohl  and  Redding,  each  message  has  four  levels.  Levels  1  and  2  deal  with  the  intentions  of 
senders,  what  is  desired  and  intended;  level  3  represents  the  ostensive  message,  what  is  actually  verbalized; 
and  level  4  delineates  the  receivers  ’  interpretations.  There  are  often  important  differences  between  levels. 
For  example,  level  1  represents  what  communicators  would  really  like  to  say,  in  the  absence  of  all 
constraints  (e.g.,  “Boss,  you  really  don’t  know  what  you’re  talking  about”);  level  2  captures  what  they 
decide  they  should  say  (“There  may  be  some  other  opinions  about  that  issue©;  level  3  depicts  what  they  do 
say  (“Sir,  that  certainly  seems  to  be  an  interesting  possibility”);  and  level  4  indicates  what  the  message 
means  to  the  receive  (“That  worker  always  is  trying  to  show  she  knows  more  than  me.”). 

2.  Shotland,  R.  (1976).  University  communication  networks:  The  small  world  methods.  New  York:  Wiley. 

“Small  World”  phenomenon  -  it  takes  approximately  6  steps  for  a  randomly  selected  person  in  one  location  to 
reach  a  randomly  selected  person  in  another.  This  phenomenon  has  been  replicated  in  several  studies  across 
cultures,  contexts  and  continents  and  has  even  become  part  of  a  national  advertising  campaign  for  a  multinational 
telecommunication  company. 

•  There  are  some  other  possible  sources  of  instruments  that  I  can  investigate.  Some  are  related  to  assessing 
change  processes  or  macro-organizational  processes.  Others  are  inventories  of  research  instruments  in  general 
and  are  not  specific  to  communication,  but  might  contain  some  communication  instruments.  Most  also  date 
from  the  late  60s  -  early  80s.  I  have  not  looked  into  any  of  them  yet.  I  have  put  an  asterisk  next  to  the 
references  I  think  might  yield  the  most  likely  possibilities.  You  can  have  someone  at  ITI  look  into  these 
references,  or  I  can  do  it.  It’s  up  to  you. 

Bonjean,  Charles  M.,  Richard  J.  Hill,  and  S.  Dale  McLemore.  (1967).  Sociological  measure:  An  inventory 
of  scales  and  indices.  San  Francisco:  Chandler. 

Kegan,  D.  L.  (1970)  Scales/RIQS:  An  inventory  of  research  instruments.  Evanston,  IL:  Northwestern 
University  Technological  Institute. 

♦Seashore,  S.,  Lawler,  E.,  Mirvis,  P.,  &  Cammann,  C.  (Eds.)  (1983).  Assessing  organizational  change.  New 
York:  Wiley. 

Shaw,  M.E.  and  J.W.  Wright.  (1967).  Scales  for  the  measurement  of  attitudes.  New  York:  McGraw-Hill. 

♦Van  de  Yen,  A.,  &  Ferry,  D.  (1980).  Measuring  and  assessing  organizations.  New  York:  Wiley. 
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Appendix  D.  DRMS  Baselining 

This  appendix  contains  the  following  documents  from  our  baselining  effort  at  DRMS: 

•  The  DRMS  cover  letter  included  in  our  mailing 

•  The  DRMS  baselining  survey  -  DRMSsurvey4.doc 

•  The  baselining  report  provided  to  DRMS. 


27 


To  all  employees: 

Communication  is  an  important  issue  in  any  organization.  This  is  especially  true  in  a  time  of  dynamic  change,  like 
the  one  that  we  are  currently  experiencing.  We,  at  HQ  DRMS  suspect  that  we  are  not  doing  a  very  good  job  in  the 
area  of  communication.  Because  of  this,  management  and  our  union  have  agreed  to  participate  in  an  on-going 
research  project,  sponsored  by  the  Air  Force,  examining  this  issue.  The  Center  for  Electronic  Commerce  (CEC)  of 
Ann  Arbor,  MI  is  conducting  this  communications  assessment  pilot  program.  This  is  not  just  another  case  of 
management  hiring  consultants  to  tell  us  what  to  do.  DRMS  is  not  funding  this  project. 

CEC  staff  will  be  conducting  two  types  of  employee  surveys.  First,  a  baseline  survey  will  ask  employees  what  they 
know  about  particular  DMRS-related  topics.  This  survey  should  take  about  10-15  minutes  to  complete.  All 
responses  will  be  sent  directly  to  the  CEC  and  will  be  completely  confidential.  No  DRMS  staff  will  ever  see  any 
individuals  completed  survey.  Once  the  data  is  collected  and  analyzed,  everyone  who  participated  will  be  appraised 
of  the  results. 

After  this  initial  survey,  some  of  you  may  also  receive  one  or  more  questionnaires  asking  about  specific  messages  or 
publications.  The  surveys  will  include  some  demographic  questions  so  that  we  can  determine  which  groups  are 
receiving  information  well  and  which  are  not.  The  data  collected  will  be  analyzed,  with  the  results  presented  back  to 
DRMS  management,  the  union,  and  all  of  you  who  responded  to  the  surveys. 

In  brief,  this  project  is  about: 

•  Are  people  getting  the  information  they  want  and  need  to  either  be  more  effective  and  comfortable  in  their  jobs, 
or  better  prepared  for  all  of  the  ongoing  changes? 

•  If  our  current  way  of  communicating  is  not  adequate,  what  do  we  need  to  do  to  correct  it? 

What’s  in  it  for  us?  This  project  is  aimed  at  improving  conditions  for  all  DRMS  employees.  The  results  and 
recommendations  of  this  pilot  should  help  all  of  us  at  DRMS  to  improve  our  internal  communications,  reduce 
misunderstandings,  and  provide  the  information  you  need  and  want  in  a  timely  manner. 

For  this  project  to  succeed  we  need  your  help.  If  you  receive  a  survey,  please  complete  it  and  return  it.  Our  goal  is 
to  achieve  a  one  hundred  percent  response  rate.  Thanks  for  your  participation  in  this  effort. 


Col.  Robert  Mansfield 
Wayne  Woosley 
Ronald  Hawkins 
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Communication  Baselining  Survey 


As  part  of  an  Air  Force  Research  Laboratory  (AFRL)  project,  your  organization  has  agreed  to  participate  in  the 
development  of  a  process  and  a  set  of  tools  to  rapidly  assess  the  effectiveness  of  communication  within 
organizations.  The  first  step  in  looking  at  communication  effectiveness  is  for  our  project  team  to  develop  an 
understanding  of  your  organization  and  it’s  current  change  efforts.  This  survey  will  provide  data  that,  when 
analyzed,  will  help  us  develop  that  understanding. 

All  of  this  survey  data  will  be  processed  at  the  Center  for  Electronic  Commerce  with  only  summary  information 
reported  back  to  your  organization.  All  specific  responses  will  be  held  in  the  strictest  confidence.  No  information 
you  provide  will  be  attributed  to  its  source  in  any  reporting  we  do  back  to  your  organization.  If  you  have  any 
comments  or  questions,  please  contact  us  using  the  information  below. 


Stanley  M.  Przybylinski 
Center  for  Electronic  Commerce 
2901  Hubbard  Road 
Ann  Arbor,  MI  48 1 05 
Phone:  (734)  769-4517 
FAX:  (734)213-3407 
Email:  smp@erim.org 


Ronald  E.  Kohler 
Center  for  Electronic  Commerce 
2901  Hubbard  Road 
or  Ann  Arbor,  MI  48 1 05 

Phone:  (734)  769-4384 
FAX:  (734)  213-3407 
Email:  rek@erim.org 


Please  complete  all  of  the  items  on  the  following  pages  to  the  best  of  your  ability.  When  you  finish  the  survey, 
please  return  it  to: 


Stanley  M.  Przybylinski 
Center  for  Electronic  Commerce 
2901  Hubbard  Road 
Ann  Arbor,  MI  48105 
Phone:  (734)  769-4517 
FAX:  (734)  213-3407 
Email:  smp@erim.org 


Thank  you  for  your  time  and  effort. 
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Background 


1 .  Work  location 

Qhqdrms., 

□  DRMO.2 

2.  How  long  have  you  worked  at  this  location? 


3.  Grade _ 

4.  How  long  have  you  worked  in  your  current  position? 


Culture 

While  our  project  focuses  on  communication ,  organizational  culture  and  communication  are  closely  related.  Many  attributes  of 
culture ,  such  as  norms  and  trust ,  directly  affect  how  people  communicate .  Please  answer  the  following  questions  about  your 
organization ’s  culture  for  the  work  location  you  designated  above. 

For  the  following  questions,  please  check  the  box  the  best  represents  your  opinion  about  the  numbered  statements: 


Strongly  Somewhat  Neither  agree  Somewhat  Strongly 

disagree.!  disagree^  nor  disagree.3  agrees  agree.5 


5. 

People  I  work  with  are  direct  and  honest 

Q 

with  each  other. 

6. 

People  I  work  with  accept  criticism 

JB 

7. 

WlulO U t  UCWUUUilg. Uwlyl wl V y*  ■.  _ ;%'r. 

People  I  work  with  resolve 

□ 

disagreements  cooperatively. 

People  I  work  with  function  as  a  team. 

■M 

9. 

People  I  work  with  are  cooperative  and 

□ 

considerate. 

id: 

People  I  work  with  constructively 
confront  problems.  ■ 

□ 

11. 

People  I  work  with  are  good  listeners. 

□ 

12. 

People  I  work  with  are  concerned  about 
eachother.  ; * ; ; / ;  f  ;  i k 

f§ 

13. 

Labor  and  management  have  a 
productive  working  relationship. 

□ 

14:  lliis  organization 

;■  -out  my  best  efforts.'’  ••  a/a'.'vV'" 

□ 

15.  This  organization  respects  its  workers. 

ri’.'’ .9., 

16.  This  organization  treats  people  in  a 

il  □ 

consistent  and  fair  manner. 

1 7.  Working  here  feels  like  being  part  of  a 

□ 

family. 

18  ’  There  is  an  atmosphere  of  trust  in  this  ; 

□ 

'  organization.  iii"' 

19.  This  organization  motivates  people  to  be 
efficient  and  productive. 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

JH 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□L 

Ivi'ii. 

□ 

HI 

v-H- 

□ 

□ 

□ 

□ 

Hi 

;  □ 

□ 

Hi 

□ 

'  □ 

□ . 

□ 

□ 

:  □ 

□ 

□ 

□ 

□ 

□ 

□ ; 

□ 

□ 

□ 

□ 

□ 

Please  turn  to  the  next  page. 


Strongly 

disagree.. 

20. 

I  get  enough  information  to  understand 
the  big  picture  here. 

□ 

21. 

When  changes  are  made  the  reasons  why 
are  made  clear. 

□ 

22. 

I  know  what's  happening  in  work  areas 
outside  of  my  own. 

□ 

23. 

I  get  the  information  I  need  to  do  my  job 
well. 

□ 

24. 

I  have  a  say  in  decisions  that  affect  my 
work. 

□ 

25. 

I  am  asked  to  make  suggestions  about 
how  to  do  my  job  better. 

26. 

This  organization  values  the  ideas  of 
workers  at  every  level. 

□ 

27. 

My  opinions  count  in  this  organization. 

28. 

My  job  requirements  are  made  clear  by 

□ 

my  supervisor. 

29. 

My  supervisor  teUs  me  whcm  1  do  a  good 

job V;>'  ■  ■":/  . 

30. 

...... 

My  supervisor  takes  criticism  well. 

□ 

31. 

My  supervisor  delegates  responsibility. 

32. 

My  supervisor  is  approachable. 

□ 

33. 

My  supervisor  gives  me  criticism  in  a 

?;o::  □ 

positive  manner. 

34. 

My  supervisor  is  a  good  listener. 

□ 

35. 

My  supervisor  tells  me  how  I'm  doing. 

36. 

Decisions  made  at  meetings  get  put  into 
action. 

□ 

37; 

Everyone  takes  part  in  discussions  at 
meetings. 

.•V  □ 

38. 

Our  discussions  in  meetings  stay  on 
track. 

□ 

39. 

Time  in  meetings  is  time  well  spent 

□ 

40. 

Meetings  tap  the  creative  potential  of 
the  people  present. 

□ 

Somewhat 

Neither  agree 

Somewhat 

Strongly 

disagree^ 

nor  disagree^ 

agrees 

agree.5 

□ 

□ 

□ 

□ 

□ 

□  ■; 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

-  :  S' 

□ 

□ 

□ 

□ 

:-;::OVyy 

□ 

□ 

□ 

□ 

□  : 

’■  Q.'  •’ 

□ 

□ 

□  ’ 

□ 

□ 

ymw-  . 

□ 

□ 

□ 

□ 

□ 

V''  ;Q;  • 

WjjfSS 

□ 

□ 

□ 

□ 

□ 

vQ;  i;:  f ; 

□ 

□ 

□ 

□ 

□ 

;  :  Qi;X;, 

y-:Q 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

Please  turn  to  the  next  page. 


Organizational  Communication 

This  is  a  series  of  questions  about  how  people  communicate  at  work.  Imagine  a  typical  week  at  work,  and  answer  the  questions 
accordingly.  Please  attempt  to  answer  all  the  questions. 

41 .  Do  you  have  subordinates  working  for  you? 

□  Yes.! 

□  No_2 

Please  indicate  the  degree  of  your  agreement  with  each  statement  by  checking  the  most  appropriate  alternative  below  each 
statement. 

42.  How  free  do  you  feel  to  discuss  with  your  immediate  superior  the  problems  and  difficulties  you  have  in  your  job  without 
jeopardizing  your  position  or  having  it  "held  against  you”  later  in  this  organizations? 

>.-Q  Very  cautious^ 

□  Somewhat  cautious_2 
;  O  It  depends^ 

□  Somewhat  ffee_4 
Q  Completely  free,* 

43.  How  often  is  your  immediate  superior  successful  in  overcoming  restrictions  (such  as  regulations  or  quotas)  in  getting  you 
the  things  you  need  in  your  job  such  as  equipment,  personnel,  etc.? 

□  Never  successful.! 

□  Very  seldom  successful  -■ 

□  Successful  half  of  the  time_3 
o  Successful  most  of  the  time_4 

□  Always  successful^ 


44.  Immediate  superiors  at  times  must  make  decisions  that  seem  to  be  against  the  interests  of  their  subordinates.  When  this 
happens  to  you  as  a  subordinate,  how  much  trust  do  you  have  that  your  immediate  superior's  decision  was  justified  by  other 
considerations? 

□  No  trust.!  ..  .  '  v-J-i 

□  A  little  trusty 

□  Trust  half  of  the  timej3 

□  Trust  most  of  the  time^ 

O  Trust  all  of  the  time_5 

45.  In  general,  how  much  do  you  feel  that  your  immediate  superior  can  do  to  further  your  career  in  this  organization? 

□  Can  do  nothing.] 

□  Can  do  a  httle.2 

□  Can  do  a  Iot.3 

46.  How  much  weight  would  your  immediate  superior's  recommendation  have  in  any  decision  that  would  affect  your  standing  in 
this  organization,  such  as  promotions,  transfers,  etc.? 

□  No  weight.] 

□  A  little  weighty 
Q  A  lot  of  weighty 

Please  turn  to  the  next  page. 
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47.  As  part  of  your  present  job  plans,  how  much  do  you  want  a  promotion  to  a  higher  position  at  some  point  in  the  future? 


o  Content  as  I  anti 

□  Somewhat  want  a  promotion^ 

□  Very  much  want  a  promotion^ 

48.  How  important  is  it  for  you  to  progress  upward? 

o  Not  at  all  important.! 

□  Somewhat  important.2 
Q  Very  important.3 

49.  To  what  extent  do  you  have  confidence  and  trust  in  your  immediate  superior  regarding  his/her  general  fairness? 

□  No  confident  and  trust.! 

Q  A  little  confident  and  trust_2;  ‘  ; 

□  Confident  and  trust  half  of  the  time.3 

□  Confident  and  trust  most  of  the  time^ 

□  Complete  confident  and  trust_5 

While  working,  what  percentage  of  the  time  do  you  spend  interacting  with: 

50.  %  Immediate  Superior? 

51.  _ %  Subordinates? 

52.  %  Peers  (others  at  same  job  level)? 

100% 

Of  the  total  time  you  engage  in  communications  while  on  the  job,  about  what  percentage  of  the  time  do  you  use  the  following 
methods  to  communicate: 

53.  _ %  Written? 

54.  %  Face-to-face? 

55.  _ %  Telephone? 

56.  _ %  Other?  (specify) 

100% 

How  accurate  is  information  that  you  receive  from: 


Never 

Inaccurate  most 

Accurate  half  of 

Accurate  most 

Always 

accurate.! 

of  the  time.2 

the  time.3 

of  the  time^ 

accurate.5 

57.  Immediate  superiors? 

□ 

'  □ 

■:  yXX\: ri 

58.  Subordinates? 

□ 

Q 

□ 

□ 

□ 

59  Peers  (others  at  your  job  level )? 

□ 

□ 

□  ' 

How  clear  and  understandable  is  information  that  you  receive  from: 

Never 

Unclear  most  of 

Clear  half  of  the 

Clear  most  of 

Always  clear_5 

clear.j 

the  time.2 

time.3 

the  time.4 

60.  Immediate  superiors? 

□ 

□ 

□ 

□  ■  : 

6 1 .  Subordinates? 

□ 

□ 

□ 

□ 

□ 

62.  Peers  (others  at  your  job  level )? 

Q 

□ 

:  □  ■ 

Please  turn  to  the  next  page. 
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63.  If  information  you  receive  is  not  clear  or  understandable,  do  you  generally  ask  for  help  or  clarification? 


□  Yes., 

QN0_2 

If  you  answered,  “No”,  please  proceed  to  question  67. 

Please  list  the  sources  you  generally  approach  for  clarification: 


64. 

% 

Immediate  Superiors? 

65. 

_ % 

Subordinates? 

66. 

_ % 

Peers  (others  at  same  job  !eve0? 

100% 


67.  How  often  do  you  find  that  the  lack  of  available  information  hinders  your  performance  in  this  organization? 
Q  Never  hinders., 

□  Seldom  hinders^  •  . 

□  Hinders  half  of  the  time.3 

•  □  Hinders  most  of  the  timeu 
Q  Always  hinders_5 

68.  How  often  do  you  feel  that  you  receive  more  information  than  you  can  efficiently  use  in  this  organization? 

□  Never  have  too  much  information., 

□  Seldom  have  too  much  infonnatioa.2 

□  Have  too  much  information  half  of  the  time.3 

□  Have  too  much  information  most  ofthe  time^ 

□  Always  have  too  much  information^ 

Of  the  total  time  you  spend  receiving  information  at  work,  what  percentage  comes  from: 

69.  _ %  Immediate  Superiors? 

70.  _ %  Subordinates? 

71.  _ _ _%  Peers  (others  at  same  job  level)? 

100% 

Of  the  total  time  you  spend  sending  information  at  work,  what  percentage  goes  to: 


72. 

_ % 

Immediate  Superiors? 

73. 

% 

,  _  Subordinates? 

74. 

_ % 

Peers  (others  at  same  job  level)? 

100% 


Please  turn  to  the  next  page. 
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When  transmitting  information  to  others  in  this  organization,  do  you  summarize  by  emphasizing  those  aspects  that  are  important 
and  minimizing  those  aspects  that  are  unimportant?  How  often  do  you  do  this  with: 


Never.! 

Seldom.2 

Half  of  the 
time.3 

Most  of 
the  time_4 

Always. 

75.  Immediate  superiors? 

□ 

□ 

■  □ 

76.  Subordinates? 

□ 

□ 

□ 

□ 

□ 

77.  Peers  (others  at  your  job  level )? 

□ 

Of  the  total  amount  of  information  you  receive 

at  work,  how  much  do  you  pass  on  to: 

None.i  Some.2  Half_3 

Mo  st_4 

An., 

78.  Immediate  superiors? 

□ 

□ 

□ 

□ 

□ 

79.  Subordinates?  ;  y;; 

80.  Peers  (others  at  your  job  level )? 

□ 

□ 

□ 

□ 

□ 

How  desirable  do  you  feel  it  is  in  your  department  to  interact  frequently  with: 

Somewhat 
desirable^ 

□ 

□  w; 

84.  Are  there  forces  that  cause  you  to  distort  information  that  you  send  upward  in  this  organization? 

G  There  are  no  forces.! 

□  There  are  very  few  forces_2 
O  There  are  lots  of  forces_3 


Very  Somewhat  Neither  desirable  or 

undesirable.!  undesirable.2  undesirable.3 

81.  Immediate  superiors?  Q  Q  Q 

82.  Subordinates?  □  □  □ 

83.  Peers  (others  ait  your  jot>  level )?  ;  Q.  V;'  Q;‘r 


85.  Put  check  the  box  next  to  the  expression  that  best  represent  how  you  feel  about  communications  in  general,  including  the 
amount  of  information  you  receive,  interaction  with  your  immediate  superior  and  others,  the  accuracy  of  information 
available,  etc. 


G  Very  positive., 

□  Positive.2 
G  Neutral.3 

□  Negative^ 

G  Very  negative.5 


Please  turn  to  the  next  page. 


Very 

desirable. 

■mm 

□ 
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Computing  Issues 

Many  of  the  changes  to  the  business  processes  within  your  organization  will  require  you  to  use  computers  more  as 
part  of  your  daily  work .  Much  of  this  work  involves  communication  of  one  form  or  another .  Please  answer  the 
following  questions  about  your  computer  usage. 

86.  Do  you  have  access  to  a  computer? 

□  Yes,  I  have  one  for  my  personal  use.  j 

Q  Yes,  1  share  one  with  one  or  more  co-workers^  , 

□  No_3 

If  you  answered  “No”,  please  proceed  to  Question  107. 

87.  What  percentage  of  your  workday  would  you  estimate  that  you  spend  using  your  computer?  _ % 

88.  Does  this  computer  provide  access  to  the  Internet? 

□  Yes.! 

□  No_2 

89.  Do  you  have  your  own  electronic  mail  address  at  work? 

■  Q  Yes., 

□  No.2 

In  the  next  few  questions  we  would  like  you  to  assess  your  skill  at  using  different  types  of  computer  applications  as  part  of your 
day-to-day  work: 

•  Applications  programs  like  Microsoft  Word,  Microsoft  Excel,  Microsoft  PowerPoint,  etc. 

•  Electronic  mail  programs  like  Microsoft  Outlook,  Eudora,  cc :  Mail,  etc . 

•  Internet  browsers  like  Netscape,  Internet  Explorer,  etc. 

•  DRMS  business  systems  like  DAISY,  MILSTRIP,  IRIS,  RTD,  etc. 

If  you  do  not  normally  use  that  type  of  application  at  work,  please  check  “Do  Not  Use  ” 


90.  Applications  program 

91.  Electronic  mail  programs 

92.  Internet  browsers 

93.  DRMS  business  systems 

For  these  same  types  of  computer  applications  we  'd  like  you  to  assess  the  level  of  training  you  have  received  to  date: 


94.  Applications  program 

95.  Electronic  mail  programs 

96.  Internet  browsers 

97.  DRMS  business  systems 

Please  turn  to  the  next  page. 


Experts 

Intermediate^ 

Average.3 

Novice^ 

Do  Not 
Use.j 

□ 

□ 

□ 

□ 

□ 

>1:  - 

'■'v'fQy 

'.VI 

□  . 

□ 

□ 

□ 

□ 

r  VO:'  '...y' 

IV.-O-V; 

IV^VQl: 
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As  the  use  of  computer  systems  becomes  more  important  in  your  every  day  operations ,  support  for  those  systems  becomes  more 
important.  The  next  few  questions  focus  on  support  for  your  computer  systems . 

We  are  interested  in  how  you  obtain  your  computer  support.  The  “Full  Time  "  and  “Part  Time  ”  options  below  are  for  people  on 
your  staff  that  provide  support .  Some  DRMOs  have  contractor  support  and  some  get  support  through  other  means. 


Full  Time., 

Part  Time_2 

Contractor.3 

Others 

98.  Hardware  Support 

□ 

□ 

□ 

□ 

99.  Software  Support 

qt;:: 

o 

In  the  next  few  questions  we  would  like  you  to  assess  the  skill  of  your  support  person  in  assisting  you  with  the  different  types  of 
computer  problems  you  may  face  as  part  of  your  day-to-day  work. 

•  Applications  programs  like  Microsoft  Word,  Microsoft  Excel,  Microsoft  PowerPoint,  etc . 

•  Electronic  mail  programs  like  Microsoft  Outlook,  Eudora,  cc:  Mail,  etc. 

•  Internet  browsers  like  Netscape,  Internet  Explorer,  etc. 

•  DRMS  business  systems  like  DAISY,  MILSTRIP,  IRIS,  RTD,  etc. 

•  Networking  issues 


Expert., 

Intermediate^ 

Average.3 

Novice^, 

Do  Not 
Use.5 

100.  Applications  program 

a 

□ 

□ 

□ 

□ 

101.  Electronic  mail  programs 

O' 

102.  Internet  browsers 

□ 

□ 

□ 

□ 

□ 

103.  DRMS  business  systems 

□  ; 

□ 

□ 

104.  Network  issues 

□ 

□ 

□ 

□ 

□ 

105.  Some  computing  support  is  provided  through  a  hot  line  to  HQ  DRMS.  For  a  typical  systems  problem,  how  long  does  it 
take  to  receive  a  response?  Please  check  one. 

□  0  to  4  hours,  i  □  1 6  to  24  hours^ 

Q  4  to  8  hours.2  Q  1  to  2  days.5 

□  8  to  16  hours.3  □  Longer  than  2  days_6 


106.  For  a  typical  systems  problem,  how  would  you  rate  the  quality  of  service  you  receive? 

□  Excellent., 

□  Good  2 

Q  Average.3 
QPoor_4 


Please  turn  to  the  next  page. 
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Computing  Issues  (con’t) 

ONLY  DRMO  Chiefs  should  complete  this  section .  If  you  do  not  currently  hold  this  position ,  please  proceed  the 
next  page. 

Computers  are  increasingly  becoming  essential  to  the  way  all  organizations  function.  Y our  organization  is  no 
exception.  Since  much  of  the  communication  that  occurs  is  computer-based,  as  part  of  our  project  we  need  to 
understand  the  pattern  of  computer  usage  within  each  DRMO.  While  the  previous  section  did  contain  questions 
about  individual  computer  usage,  we  are  only  sampling  from  the  DRMO  staff.  Hopefully,  all  DRMO  chiefs  will 
complete  this  section. 

There  are  several  pieces  of  information  we  would  like  to  obtain:  how  many  computers  you  have,  how  many  are  used 
primarily  by  one  person  vs.  shared,  how  many  people  use  email,  and  how  many  of  your  machines  have  Internet 
access.  For  example,  if  your  DRMO  has  20  people,  10  computers  (5  used  mostly  by  one  person  and  5  in  more  public 
areas),  one  work-related  individual  Internet  email  account,  and  one  computer  with  Internet  access,  your  table 
would  look  like: 


#  of  People 

Total  Number 
of  Computers 

Individual  Email 
Accounts 

Internet 

Access 

Individual 

Shared 

Access 

20 

5 

5 

1 

1 

Please  enter  a  number  into  each  cell.  Approximate  answers  are  acceptable. 


#  of  People 

Total  Number 
of  Computers 

Individual  Email 
Accounts 

Internet 

Access 

Individual 

Shared 

Access 

107. 

108. 

109. 

110. 

111. 

Please  turn  to  the  next  page. 
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Enterprise  Management 

The  following  questions  are  intended  to  gauge  your  understanding  of  the  principles  of  Enterprise  Management  and 
the  changes  DRMS  has  undergone  over  the  last  several  years.  New  policies ,  procedures ,  and  processes  have  been 
and  will  continue  to  be  put  into  place .  For  DRMS  to  succeed,  all  employees  must  be  on  the  “same  page  ”  about 
these  efforts . 

The  questions  in  this  section  are  all  multiple  choice.  For  each  statement,  please  check  the  box  to  the  right  of  the 
numbered  item  only  if  you  think  it  applies  to  that  statement. 

Please  complete  all  items. 

Definitions 

Enterprise  Management  (EM)  is:  Please  check  all  that  apply 

£  1.  Anotiier  name  for  downsizing.  .  \  Q; 

2.  An  approach  to  use  market  factors  to  determine  what  to  privatize.  □ 


Privatization  and  outsourcing  decisions  will:  Please  check  all  that  apply. 

the  result  of  political  or  bureaucratic  forces.  O 

5.  Occur  only  when  they  can  lower  cost  and  increase  performance.  □ 

6.  Be  based  on  an  analysis  of  strategic  and  economic  factors.  O 

7.  Turn  contractors  who  take  over  DRMS  functions  into  our  competitors.  □ 

8.  Not  result  in  lost  Government  jobs.  Q 


The  goals  of  Enterprise  Management  include: 


Please  check  all  that  apply. 


9.  Reducing  infrastructure  by  i/3  by  the  end  of  FY  98. 

10.  Giving  customers  a  choice  in  service  providers. 

11.  Reducing  overhead  by  10%  in  FY  97  and  15%  in  FY  98. 

12.  Introducing  the  private  sector  into  DRMS  activities  where  mature  commercial 
markets  exist. 

13.  Maximizing  accountability  and  results  for  Government  services. 


□ 

□ 

□ 

□ 

□ 


If  we  are  successful  with  Enterprise  Management:  Please  check  all  that  apply. 


14.  Government  employees  will  perform  aHUtility  services.  □ 

15.  The  private  sector  will  be  introduced  into  those  activities  of  the  DRMS  mission  q 

where  mature  commercial  markets  exist. 

1 6.  Marketplace  services  will  all  be  contracted  out  Q 

17.  Privatization  and  outsourcing  will  occur  only  when  they  can  lower  cost  and  q 

'  increase  performance. 


Please  turn  the  page. 
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Mechanisms 

Implementing  the  philosophies  of  Enterprise  Management  required  DRMS  to  identify,  test,  and  apply  several  new 
approaches  to  the  planning,  management,  and  execution  of  DRMS  business  functions. 

Our  new  business  processes  will:  Please  check  all  that  apply. 


1 8.  Ensure  that  all  Utility  services  are  performed  by  Government  employees.  O 

19.  Help  DRMS  become  a  “knowledge-based”  organization.  _  . . □ . . 

20.  Include  the  use  of  service  contracts,  sales  contracts,  and  public/private  q 

' partnerships.  v  ^  .  -  -  '  <  r  *  -  **. 

21.  Contract  out  all  Marketplace  services. 

22.  Practically  elinnnate  pl^sical  handling  and  transporting  property  by  DRMOs.  □ 


Activity  Based  Costing  (ABC)  will  help  DRMS: 


Please  check  all  that  apply . 


23.  Determine  which  DRMOs  to  close. 

24.  Management  to  identify  specific  positions  to  target  for  RIF  actions. 

25.  Measure  the  true  cost  of  performing  activities  to  produce  products  and  services. 

26.  Identify  how  human  and  financial  resources  are  consumed. 


Many  of  these  initiatives  are  described  in  the  EM  News,  a  publication  created  to  increase  awareness  among  DRMS 
employees. 

The  EM  News:  Please  check  all  that  apply. 

27.  Is  FAXed  to  each  DWdO  and  copied  each  en^loyep;  -  ;  Q 

28.  Is  available  on  the  World  Wide  Web  (WWW).  Q 

29  ; deludes  a  bulletin  board  system  to  ask  questions  and  provide  feedback^  :  V  Q 

30.  Provides  a  phone  and  FAX  number  for  those  who  do  not  have  computer  access.  □ 


The  use  of  Office  of  the  Management  and  Budget  (OMB)  Circular  A-76,  “Performance  of  Commercial  Activities”: 
Please  check  all  that  apply. 

31.  WiH  first  be  applied  within  DLA  to  both  the  Defense  Distribution  Center  and  Q 

32.  Will  be  directed  by  an  outside  contractor.  Q 

33.  Is  to  conduct  managed  competitions  between  the  public  and  private  sector.  ■  v 

34.  Will  initially  focus  on  selected  processes  at  1 0  DRMOs.  Q 

’  35;  WfllfeledtyDR^TOfhcqnt^ 

36.  First  occurred  during  the  Eisenhower  administration.  Q 

37.  Will  only  be  to  privatize  DM^  -\'r  •  Q  ;'V 

38.  Always  results  in  a  process  being  “contracted  out”.  Q 

39.  Always  results  in  die  loss  of  some  Government  jobs.  Q 


Please  turn  the  page. 
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As  part  of  the  new  Recycling  Control  Point  (RCP)  definition:  Please  check  all  that  apply. 


40.  Property  remains  at  the  Depots  to  save  labor  and  other  costs.  @1 

4 1 .  Several  DRMOs  may  no  longer  perform  this  function.  □ 

42.  The  receipt,  screening,  and  sales  processes  are  automated.  Vv.;Q-  • 

43  DRMS-CR  (Battle  Creek)  acts  as  a  “virtual  DRMO”.  □ 

.  44.  The  Depot  retains  physical  custody  of  the  property  throughout  the  R/T/D/S  cycles.  O 

45.  The  World  Wide  Web  will  be  an  essential  part  of  this  function.  □ 


Service  Centers 

In  mid-April,  DRMS  opened  two  new  HQ  offices  to  support  CONUS  field  activities.  These  “Support  Centers”  will: 
Please  check  all  that  apply. 

46.  Have  dedicated  personnel  who  are  knowledgeable  about  all  part  of  our  business. 

47.  Be  staffed  by  HQ  personnel  with  little  or  no  field  experience.  □ 

4&  O^^OCUS to 

49.  Provide  support  and  technical  guidance  to  the  Zone  Managers  and  DRMOs  in  the  q 

day-to-day  business  of  field  offices. 

50.  Provide  access  to  experts  from  other  HQ  DRMS  areas.  Q 

5 1 .  Eventually  handle  most  of  the  information  flow  to  and  from  the  field.  □ 


Thank  you  for  your  time  and  effort. 
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Communication  Baseline  Survey  Results: 
Defense  Reutilization  Marketing  Service  (DRMS) 

ERIM 


Center  for  Electronic  Commerce 
December  1998 


The  Center  for  Electronic  Commerce  recently  administered  a  survey  to  members  of  the  Defense  Reutilization  and 
Marketing  Service  (DRMS).  The  survey  consisted  of  questions  about  culture,  communication,  computing  issues, 
and  the  implementation  of  the  Enterprise  Management  (EM),  including  both  structure  and  processes.  The  focus  of 
investigation  was  the  communication  within  the  Marketplace  group  and  those  individuals  at  Defense  Reutilization 
and  Marketing  Offices  (DRMOs)  that  communicate  most  with  this  HQ  organization.  The  data  collected  will  serve  to 
provide  a  baseline  about  current  knowledge  and  communications  practices  as  part  of  a  pilot  program  funded  by  the 
Air  Force  Research  Laboratory  (AFRL/HESS).  We  would  like  to  thank  the  HQ  DRMS  staff  who  assisted  us  and  all 
of  our  respondents  for  their  time  and  effort. 

Interpreting  the  Results 

The  complete  baselining  report  for  DRMS  consists  of  three  elements: 

1 .  This  summary  document 

2.  A  set  of  tables  entitled  “DRMS  Communication  Baselining  Survey  Summary  Statistics”  that  provide  summary 
statistics  for  all  of  the  questions.  (These  are  NOT  included  in  this  appendix.) 

3.  A  copy  of  the  original  survey  for  reference. 

After  a  section  with  “General  Comments”,  the  sections  of  this  summary  match  the  section  headings  of  the  baselining 
questionnaire.  In  some  cases  questions  may  seem  to  apply  to  more  than  one  heading.  For  example,  there  are  some 
questions  under  the  heading  of  “Organizational  Communication”  that  are  related  to  culture.  For  the  purposes  of  this 
summary,  the  analysis  of  all  questions  will  occur  under  the  headings  in  the  original  survey. 

In  the  summary  statistics  document,  summary  statistics  for  each  question  are  listed  by  question  number.  Statistics 
were  calculated  for  seven  different  categories: 

1.  Headquarters  (HQ)  (100  surveys  distributed) 

2.  All  DRMOs  (34  DRMOs  surveyed  in  all) 

3.  DRMOs  surveyed  in  the  East  region  (East)  (17  in  all) 

4.  DRMOs  surveyed  in  the  West  region  (West)  (17  in  all) 

5.  Small  DRMOs  (3  in  East,  4  in  West;  from  5  to  1 1  surveys  distributed  per  DRMO)1 

6.  Medium  DRMOs  (5  in  East,  6  in  West;  from  8  to  10  surveys  distributed  per  DRMO) 

7.  Large  DRMOs  (9  in  East,  7  in  West;  from  8  to  39  surveys  distributed  per  DRMO) 

The  letter  N  represents  the  number  of  usable  responses  we  received  for  that  item.  Next  to  that  number  we  show  the 
mean  (or  arithmetic  average)  of  the  responses  to  that  question.  In  some  cases,  several  items  are  also  combined  to 
create  a  score  for  some  attribute  of  your  organization,  such  teamwork,  and  information  flow.  Also,  please  note  that 
results  are  presently  somewhat  differently  for  several  questions.  Questions  41,  63,  88,  and  89  were  “Yes/No” 
questions.  The  notation  “Y/N/NR”  next  to  the  question  number  indicates  that  the  three  summary  statistics  for  that 
question  for  the  number  of  “Yes”,  “No”,  and  “Non-Responses”.  For  example,  when  question  41  was  posed  to  HQ 
respondents,  12  answered  “Yes”,  45  answered  “No”,  and  there  was  1  “Non-Response”. 


1  Total  DRMO  staff,  not  number  of  respondents,  determined  “Size”. 
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For  most  of  the  “Yes/No”  questions  we  included  the  actual  number  of  responses,  i.e.,  if  21  people  answered  a 
question  “Yes”,  the  number  “21”  is  listed  next  to  that  response.  In  the  “Computing  Issues”  section,  we  listed  the 
actual  number  of  responses  to  questions  about  computer  usage  and  training  (items  86  -  106). 

The  “Enterprise  Management”  section  was  intended  to  test  the  knowledge  of  the  DRMS  staff  about  the  new  EM 
structure,  policies,  and  processes.  As  such,  these  questions  included  both  true  and  false  statements.  In  the  copy  of 
the  survey  attached,  the  TRUE  statements  are  shown  in  bold.  On  the  summary  statistics  report,  the  true  statements 
are  shown  in  reverse,  i.e.,  white  text  on  black  background.  The  total  number  of  surveys  processed  for  each  category 
is  listed  in  the  N  column  next  to  questions  1, 4,  and  39. 

It  is  important  to  note  that  these  summary  statements  might  also  reflect  our  project  team’s  on-site  experience  at 
DRMS.  In  many  cases,  the  survey  analyses  served  to  reinforce  the  impressions  we  developed  during  our  time  on¬ 
site. 

General  Comments 

Both  HQ  DRMS  and  DRMO  staff  were  surveyed.  HQ  DRMS  distributed  100  surveys  to  HQ  staff  members.  The 
completed  surveys  were  collected  by  HQ  DRMS  and  returned.  The  CEC  mailed  packets  to  34  selected  DRMOs. 
These  DRMOs  were  selected  to  get  a  range  of  small,  medium  and  large  sites  in  both  the  East  and  West  regions. 
(International  DRMOs  were  intentionally  omitted  from  this  effort.)  These  packets  contained  enough  surveys  for  the 
DRMO  chief  and  staff  from  the  following  job  series:  1 104,  1107,  028,  and  029.  These  staff  members  were  chosen 
because  of  their  need  to  communicate  with  the  Marketplace  group.  Each  DRMO  respondent  received  a  cover  letter 
explaining  the  purpose  of  the  survey,  a  blank  copy  of  the  questionnaire,  and  a  postage-paid  envelope  addressed  to  the 
CEC.  Follow-up  calls  were  made  to  each  DRMO  to  ensure  that  the  packets  were  received.  Two  DRMOs  required  a 
second  packet. 

The  table  below  lists  the  response  rate  by  the  seven  categories  above. 


HQ 

All 

East 

West 

Small 

Medium 

58% 

60% 

61% 

60% 

64% 

57% 

61% 

This  is  an  excellent  response  rate  for  a  survey  of  this  type.  It  is  important  to  note  that  those  who  chose  to  answer  the 
questionnaire  may  differ  significantly  from  those  who  did  not,  which  could  bias  the  results.  In  the  future,  special 
efforts  to  increase  the  response  rate  of  such  surveys  should  be  taken  to  ensure  that  the  results  are  indeed 
representative  of  the  DRMS  population. 

Most  of  our  rating  scales  included  5  options  with  the  middle  option  (3)  being  either  “Neither  agree  nor  disagree”, 
“Neither  desirable  or  undesirable”,  or  “Half  of  the  time”.  For  many  of  these  questions,  “3”  was  the  most  common 
choice.  While  this  could  mean  that  the  respondent  indeed  had  no  real  opinion,  the  middle  choice  is  often  chosen 
routinely  as  a  way  to  quickly  complete  a  survey.  This  could  indicate  either  apathy  (they  didn’t  care  or  just  marked 
the  center  for  speed)  or  fear  of  repercussions.  As  we  stated  in  the  cover  letter,  all  responses  will  continue  to  be 
confidential,  with  only  summary  data  provided. 

If  you  scan  across  almost  most  of  the  questions,  two  patterns  are  clear.  People  in  small  organizations  consistently 
gave  higher  scores  that  those  in  medium-sized  or  large  organizations  (both  DRMOs  and  HQ).  Since  our  focus  is  on 
communication,  this  result  makes  sense.  It  is  much  easier  to  keep  the  lines  of  communication  open  and  effective  if 
there  are  fewer  people  participating.  The  other  result  is  that  on  almost  every  question  or  measure  (items  5  through 
85),  DRMOs  in  the  East  scored  equal  to  or  higher  than  those  in  the  West. 

There  are  several  groups  of  questions  about  interaction  groups  (items  50-52),  communication  methods  (53-56), 
sources  of  clarifying  information  (64-66),  and  the  receipt  (69-71)  and  sending  of  information  (72-74).  All  of  these 
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groups  required  the  respondents  to  provide  a  set  of  numbers  that  totaled  to  100(%).  This  proved  problematic  in 
many  cases.  In  spite  of  their  best  intentions,  5-10%  of  the  respondents  could  not  seem  to  add  to  100. 

Background 

The  summary  statistics  for  questions  2  and  4  (tenure  at  the  current  location  and  current  position,  respectively)  show 
significant  difference  between  HQ  and  DRMO  respondents.  On  average,  DRMO  respondents  have  worked  almost 
twice  as  long  at  their  current  postings  that  their  counterparts  at  headquarters  and  have  been  in  their  current  positions 
almost  three  times  as  long.  This  could  contribute  to  some  of  the  frustration  that  DRMO  staff  has  expressed  with  HQ. 

Culture 

The  “Culture”  section  of  this  survey  was  derived  from  an  existing  questionnaire.  In  that  questionnaire,  the  groups  of 
questions  in  this  section  represented  different  higher  level  concepts : 

•  Questions  5  - 12  -  Teamwork 

•  Questions  13  -  19  -  Morale 

•  Questions  20  -  23  -  Information  Flow 

•  Questions  24  -  27  -  Involvement 

•  Questions  28  -  35  -  Supervision 

•  Questions  36  -  40  -  Meetings 

The  ratings  scales  are  from  1-5,  with  1  being  the  low  score.  With  this  system,  higher  scores  are  better  both  on 
individual  questions  and  for  each  concept.  While  the  concept  scores  are  important,  the  answers  for  the  individual 
items  can  help  indicate  specific  areas  for  improvement.  For  example,  answers  for  questions  36  through  40  can 
pinpoint  how  your  staff  believes  meetings  can  best  be  improved. 

We’ll  report  on  each  concept  in  turn. 

Teamwork:  The  items  ask  about  honesty,  problem  resolution,  and  team  interactions.  Teamwork  rated  highest  in 
small  DRMOs,  with  HQ  a  close  second.  The  lowest  item  across  all  seven  categories,  ability  to  accept  criticism 
without  becoming  defensive  (ranging  from  2.4  to  2.9)  is  an  area  of  concern. 

Morale:  These  questions  focus  on  working  relationships,  motivation,  trust,  and  respect.  Morale  is  very  low  for  all 
categories  except  small  DRMOs,  with  almost  all  of  the  six  scale  items  scoring  below  three.  Trust  is  particularly  low 
(item  18).  In  a  time  of  great  change,  with  many  people  fearful  for  their  jobs,  this  is  understandable. 

Information  Flow:  The  questions  on  information  flow  are  general  in  this  section,  asking  about  quantity,  quality,  and 

accessibility  of  information.  (Information  flow  is  also  considered  in  more  detail  in  the  Organizational 

Communication  section.)  Small  and  medium-sized  DRMOs  did  the  best  here  and  HQ  the  lowest.  Given  that  HQ  is 
in  one  location  and  has  access  to  more  communication  technology,  this  low  score  is  of  concern. 

Involvement:  This  concept  includes  both  whether  people  can  participate  and  if  their  actions  are  valued.  Again,  the 
overall  score  is  below  3.  While  people  say  they  may  be  asked  for  their  input  (items  25  and  26),  they  feel  that  their 
input  does  not  count  (items  27  and  28). 

Supervision:  This  concept  focuses  on  aspects  for  the  respondents’  immediate  supervisor.  This  is  the  highest  rated 
concept  across  just  about  all  categories. 

Meetings:  Questions  here  ask  about  attributes  of  successful  meetings  at  any  organization,  such  as  meetings  stay  on 
track,  everyone  participates,  and  decisions  made  at  the  meeting  are  enacted.  This  was  lowest  for  HQ  (2.69)  and 
highest  for  small  DRMOs  (3.45).  The  question  on  meeting  time  not  being  well  spent  was  the  lowest  item  for  all 
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categories,  with  HQ  the  lowest  at  2.4  out  of  5.  In  the  field,  decisions  made  at  meetings  are  put  into  action,  surely  a 
contributing  factor  to  the  respondents  overall  satisfaction  with  the  process. 

Organizational  Communication 

This  section  of  the  questionnaire  was  also  based  on  an  existing  survey.  While  these  items  are  not  grouped  into 
“concepts”  as  in  the  “Culture”  section,  there  are  some  natural  groupings.  For  example,  items  42  -  49  focus  on  the 
respondent’s  relationship  with  their  immediate  supervisor.  Specific  concerns  include  retribution,  advancement,  and 
trust. 

Most  employees  do  not  particularly  fear  retribution  (item  42),  which  can  be  a  good  sign.  Supervisors  are  somewhat 
successful  when  trying  to  help  you  get  what  you  need  to  do  your  job  (item  43)  and  you  have  some  trust  in  their 
decisions  (item  44).  Managers  at  small  and  medium-sized  DRMOs  do  best  at  this.  People  want  to  get  promoted 
(items  47  and  48),  but  are  not  always  sure  that  their  supervisors  can  help  (items  45  and  48). 

It  is  most  desirable  for  you  to  interact  with  your  peers  and  your  subordinates,  and  less  so  with  your  superiors  (items 
81  -  83).  Who  you  communicate  with  (items  50  -  52)  varies  by  your  location.  At  HQ,  people  spend  most  of  their 
time  communicating  with  their  peers  (59.7%).  The  effect  is  less  strong  in  the  field,  particularly  at  small  DRMOs. 
Information  sources  (items  69-71)  also  vary  by  location.  At  HQ,  people  get  most  of  their  information  their  peers 
(53.6%)  with  superiors  a  distant  second.  At  DRMOs,  it  is  just  the  opposite.  For  almost  every  category,  superiors 
provide  most  information  with  peers  trailing.  Your  communication  at  work  is  generally  either  face-to-face  or  on  the 
telephone  (items  53-56),  although  written  communication  is  used  more  at  HQ  than  in  the  field.  There  is  some  email 
use,  but  it  is  not  high  considering  how  much  you  use  your  computers  (see  the  next  section).  The  information  you  get 
from  both  above  and  below  your  level  is  accurate  and  understandable  more  than  half  of  the  time  (items  57  -  62).  If 
you  have  a  question  (items  63  -  66)  almost  all  of  you  ask  it.  Who  you  ask  is  basically  the  same  as  who  you 
communicate  with  in  general:  at  HQ  you  are  most  likely  to  ask  your  peers  and  in  the  field  your  superiors.  While  you 
like  what  you  get,  you  may  not  be  getting  enough  of  what  you  need.  The  score  on  item  67  indicates  that  you  are 
hindered  in  your  job  nearly  half  of  the  time  by  lack  of  information.  On  item  68,  you  said  you  seldom  have  too  much 
information. 

When  you  distribute  information  you  tend  to  filter  what  you  send  more  than  half  the  time  (items  75  -  77).  Item  84 
asked  about  whether  you  distort  information,  certainly  a  much  stronger  word  than  what  we  used  in  items  75  to  77 
(summarizing  important  aspects  and  minimizing  unimportant  ones).  And,  when  we  put  it  this  way,  you  said  there  are 
few  forces  that  might  cause  you  to  distort  information  you  send  to  your  superiors. 

Overall,  you  rated  your  communications  slightly  better  than  neutral  (item  85).  This  is  somewhat  confusing  since 
when  most  of  the  components  were  asked  about  separately  they  received  higher  scores.  One  possible  explanation  for 
this  is  the  low  morale  rating.  When  people  are  asked  about  specific  things,  they  can  often  focus  on  those  things 
without  being  colored  by  broader  issues,  like  morale.  When  asked  a  general  question,  like  about  overall 
communication,  they  relate  that  to  “overall”  issues,  again  like  morale. 

Computing  Issues 

DRMS  has  undergone  many  major  changes  over  the  last  few  years  and  all  of  these  contained  a  significant 
information  technology  component.  The  use  of  computers  will  become  increasingly  central  to  the  way  you  do 
business.  This  includes  as  modes  of  communication.  As  such,  we  thought  it  was  important  to  include  in  our 
baselining  an  overview  of  your  computer  access,  usage,  knowledge,  and  training. 

Most  of  the  respondents  have  a  computer  for  their  personal  use,  with  their  own  email  address  and  access  to  the 
Internet.  People  in  the  field  are  spending,  on  average,  more  than  half  their  workday  at  their  machine.  At  HQ,  this 
number  increases  to  72.9%.  Overall,  skill  level  seems  adequate.  If  many  of  your  systems  are  moving  to  the  Internet, 
the  comparatively  lower  scores  for  Internet  browsers  indicates  training  could  be  useful.  Most  of  you  believe  you 
could  benefit  from  more  training.  An  important  area  of  concern  is  lack  of  training  on  DRMS  business  systems.  This 
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has  implications  both  in  the  short  term,  i.e.,  the  need  to  provide  mode  training,  and  in  the  longer  term  are  more  and 
more  of  these  systems  are  re-implemented  for  use  on  the  Internet. 

Enterprise  Management 

Part  of  any  change  effort  is  building  an  understanding  of  the  new  structure,  policies,  and  processes.  The  questions  in 
this  section  were  constructed  by  combining: 

•  Facts  taken  from  newsletters,  manuals,  briefings,  and  other  materials 

•  Popular  misconceptions  about  the  change  provided  by  HQ  DRMS  staff. 

These  questions  were  placed  into  three  categories: 

•  Definitions  -  These  questions  about  the  tenets  of  Enterprise  Management  are  akin  to  your  new  “vision”  or 
“mission”. 

•  Mechanisms  and  Service  Centers  -  If  the  definitions  are  your  new  mission,  these  two  categories  represent 
“tactics”  chosen  to  help  your  achieve  that  mission. 

We  decided  to  report  your  “scores”  in  several  different  ways.  First,  the  summary  statistics  report  shows  the  raw 
number  of  answers  to  each  question.  Items  were  scored  with  a  “1”  if  they  were  checked  and  a  “0  if  they  were  not. 
The  “N”  listed  in  the  first  question  on  each  page  shows  how  many  surveys  were  processed  for  each  category.  For 
example,  of  the  295  DRMO  surveys  processed,  209  respondents  correctly  answered  that  Enterprise  Management  is 
“an  approach  to  use  market  forces  to  determine  what  to  privatize”. 

We  also  scored  the  results  as  a  percentage,  as  on  a  test  in  school,  from  0%  to  100%.  First  let’s  look  at  the  scores  for 
the  different  respondent  groups.  The  table  below  lists  the  percentage  of  respondents  in  each  group  that  scored  60% 
(a  “D-“)  or  less,  i.e.,  if  the  respondents  were  students,  these  numbers  show  the  percentage  of  students  who  “failed” 
the  Enterprise  Management  “exam”. 


HQ 

All 

East 

West 

Small 

Medium 

Large 

Overall 

60% 

68% 

61% 

74% 

66% 

57% 

72% 

Definitions 

48% 

59% 

56% 

62% 

53% 

51% 

63% 

Mechanisms 

69% 

76% 

73% 

78% 

75% 

71% 

77% 

Service  Centers 

57% 

63% 

59% 

67% 

59% 

62% 

65% 

These  low  scores  force  us  to  ask  some  questions  about  our  questions. 

•  Are  these  indeed  the  “right”  facts?  If  these  are  not  right  questions  to  ask  about  Enterprise  Management,  then  the 
results  do  not  mean  much.  As  people  from  outside  the  organization,  we  did  as  much  as  we  could  to  make  sure 
we  asked  about  things  that  DRMS  staff  at  least  had  the  chance  to  know.  We  read  all  of  the  newsletters, 
reviewed  briefings,  your  external  and  internal  Web  sites,  and  just  about  anything  we  could  get  our  hands  on. 

•  Is  knowledge  of  this  information  necessary  for  DRMS  to  succeed  in  changing  to  Enterprise  Management?  For 
those  affected  by  the  changes,  much  of  this  information  is  necessary.  Even  if  it  is  necessary,  however,  the  low 
scores  are  probably  related  to  morale.  Anecdotally  this  is  confirmed  by  the  people  who  sent  us  questionnaires 
with  only  one  or  two  questions  answered,  or  those  whose  chose  to  add  written  comments,  most  of  them 
extremely  negative. 

We  also  did  this  graphically  in  two  ways  for  each  respondent  group: 
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•  Overall  for  all  5 1  items 

•  By  the  categories  of  questions  in  the  Enterprise  Management  section. 


HQ  Results 


As  we  said  in  the  previous  table,  40%  of  HQ 
DRMS  respondents  scored  60  or  below.  While 
normally  this  would  be  a  low  score,  this 
represents  the  best  performance  of  any  of  the 
subgroups  analyzed. 

Because  HQ  is  the  source  for  much  of  this 
information,  it  makes  sense  that  staff  members 
there  would  know  more  about  it  than  their  peers 
in  the  field. 

Looking  at  the  results  by  question  area,  the  HQ 
group  did  the  best  on  definitions  and  worst  on 
the  mechanisms.  As  the  group  responsible  for 
implementing  many  of  these  ‘"tactics”,  this  is  of 
some  concern.  The  low  scores  on  the  service 
center  concept  makes  sense  since  HQ  staff  are 
the  providers  of  the  service,  not  the  users.  Only 
those  actually  involved  with  Service  Centers 
may  pay  attention  to  the  details. 


Results  by  Question  Area  -  HQ 


j  S3  Definitions  O  Mechanisms  m  Service  Centers ! 


co  xr  in  <o 

Percent  correct 
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DRMO  Results 


With  68%  of  the  respondents  scoring  below 
60%,  it  is  clear  that  DRMO  staff  members  did 
not  perform  well  on  this  section.  Whether  it  is 
because  morale  is  low,  or  they  did  not  want  to 
participate  in  yet  another  survey,  or  they  just  do 
not  know  the  information  is  not  clear. 

However,  if  any  of  these  explanations  are  true, 
they  ail  can  still  have  the  same  negative  effects 
on  DRMS  achieving  it’s  Enterprise 
Management  goals. 

DRMOs  scored  best  on  the  definitions  but  the 
mechanisms  were  not  well  understood.  Scores 
on  the  Service  Center  concept  were  relatively 
flat  across  all  respondents.  Since  Service 
Centers  are  an  important  part  of  making  HQ 
more  responsive  to  the  field  and  helping  the 
DRMOs  be  more  effective,  this  is  an  important 
issue. 


Overall  Results  *  AH 


m  tn  jv.  to 

Percent  correct 


Results  by  Question  Area  -  All  DRMOs 


t  b  Definitions  o  Mechanisms  a  Service  Centers 
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East  Region  DRMOs 


The  numbers  for  the  rest  of  questionnaire 
suggest  that  there  is  something  different 
between  DRMOs  in  the  East  vs.  the  West. 
The  East  had  61%  score  below  60,  the  West 
74%. 


If  you  look  across  the  all  of  the  summary 
statistics,  the  numbers  in  the  East  are  just  a 
little  bit  higher  for  each  question  area.  The 
variation  that  might  explain  this  difference  in 
score  is  on  questions  72  through  74.  People 
in  the  East  send  more  communication  upward 
to  their  superiors  than  in  the  West.  This  is 
true  although  there  is  no  real  difference  in 
how  respondents  in  each  region  say  they  feel 
about  their  superiors. 


Results  by  Question  Area  -  East 


m  Definitions  &  Mechanisms  *  Service  Centers  j 
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West  Region  DRMOs 


Respondents  in  the  West  had  most  scores  60  or 
below  at  74%.  They  did  best  on  the  definitions, 
even  though  that  was  only  in  the  5 1  -60%  range. 
Again,  their  knowledge  of  Service  Centers  was 
more  or  less  flat,  an  area  of  concern. 


Overall  Results  -  West 


Results  by  Question  Area  -  West 


i  a  Definitions  53  Mechanisms  ■  Service  Centers 
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Small  DRMOs 


For  many  things  size  matters  and  that  is  true 
also  of  communications.  While  the  small 
DRMOs  scored  the  best  in  the  other  areas, 
they  were  in  the  middle  on  knowledge  of 
Enterprise  Management.  They  scored  better 
on  Definitions  and  Service  Centers  than  the 
other  size  categories.  Their  score  on 
Mechanisms  was  almost  as  high  as  for  large 
centers.  One  lesson  from  this  is  that  while 


communication  may  be  better  overall  within 
smaller  centers,  that  does  not  carry  over  to 
communication  about  this  particular  change 


Results  by  Question  Area  -  Small 


j  a  Definitions  12  Mechanisms  ■  Service  Centers 
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Medium-Sized  DRMOs 


More  respondents  at  medium-sized  DRMOs 
“passed”  than  in  the  other  size  categories  and 
they  knew  the  most  about  definitions,  the 
“vision/mission”  of  Enterprise  Management. 
They  were  in  the  middle  on  Mechanisms  and 
Sendee  Centers. 


Results  by  Question  Area  -  Medium 
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Large  DRMOs 


As  was  stated  earlier,  large  DRMOs  scored 
lower  on  most  communication  issues.  Size 
does  matter  in  communication.  The  more 
people  in  the  mix,  the  harder  it  is  to  get  the 
message  out  and  to  maintain  clarity.  This  was 
seems  to  he  the  case  for  the  Enterprise 
Management  message.  This  group  scored  the 
lowest  overall  of  any  of  the  size  classes.  It 
also  scored  lowest  in  every  question  category. 
If  DRMSs  plan  include  consolidating  some 
smaller  DRMOs  into  the  larger  ones  where 
communication  is  clearly  ineffective,  then  the 
plans  should  include  efforts  to  addrress  these 
issues. 
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Results  by  Question  Area  -  Large 
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For  More  Information 

Please  contact:  Stanley  M.  Przybylinski 

ERIM 

Center  for  Electronic  Commerce 
2901  Hubbard  Road 
Ann  Arbor,  MI  48105 
Phone:  (734)  769-4517 
FAX:  (734)  213-3405 
Email:  smp@erim.org 

Thank  you  again  for  the  time  and  effort  you  committed  to  this  survey. 
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Appendix  E.  WR-ALC  Baselining 

This  appendix  contains  the  following  documents  from  our  baselining  effort  at  WR-ALC: 


•  The  WSSC  baselining  survey 

•  The  F-l 5  survey  results 

•  The  C- 1 4 1  survey  results 
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Communication  Baselining  Survey 

As  part  of  an  Air  Force  Research  Laboratory  (AFRL)  project,  your  organization  has  agreed  to  participate  in  the 
development  of  a  process  and  a  set  of  tools  to  rapidly  assess  the  effectiveness  of  communication  within 
organizations.  The  first  step  in  looking  at  communication  effectiveness  is  for  our  project  team  to  develop  an 
understanding  of  your  organization  and  it’s  current  change  efforts.  This  survey  will  provide  data  that,  when 
analyzed,  will  help  us  develop  that  understanding. 

All  of  this  survey  data  will  be  processed  at  the  Center  for  Electronic  Commerce  with  only  summary  information 
reported  back  to  your  organization.  All  specific  responses  will  be  held  in  the  strictest  confidence.  No  information 
you  provide  will  be  attributed  to  its  source  in  any  reporting  we  do  back  to  your  organization.  If  you  have  any 
comments  or  questions,  please  contact  us  using  the  information  below. 


Stanley  M.  Przybylinski 
Center  for  Electronic  Commerce 
2901  Hubbard  Road 
Ann  Arbor,  MI  48105 
Phone:  (734)  769-4517 
FAX:  (734)  213-3405 
Email:  smp@iti.org 


Ronald  E.  Kohler 
Center  for  Electronic  Commerce 
2901  Hubbard  Road 
or  Ann  Arbor,  MI  48 1 05 
Phone:  (734)  769-4384 
FAX:  (734)  213-3405 
Email:  rek@iti.org 


Please  complete  all  of  the  items  on  the  following  pages  to  the  best  of  your  ability.  When  you  finish  the  survey, 
please  return  it  to: 


Stanley  M.  Przybylinski 
Center  for  Electronic  Commerce 
2901  Hubbard  Road 
Ann  Arbor,  MI  48 1 05 
Phone:  (734)  769-4517 
FAX:  (734)213-3405 
Email:  snp@iti.org 


Thank  you  for  your  time  and  effort. 


Background 

1 .  Work  location 

□  F-15  WSSC., 

□  C-141  WSSC.2 

2.  How  long  have  you  worked  at  this  location? _ 

3.  Grade  _ 

4.  How  long  have  you  worked  in  your  current  position? _ 

5.  Are  you  a  member  of  any  type  of  union  organization? 

□  Yes., 

□  No.  2 

Culture 

While  our  project  focuses  on  communication ,  organizational  culture  and  communication  are  closely  related '  Many  attributes  of 
culture ;  such  as  norms  and  trust,  directly  affect  how  people  communicate.  Please  answer  the  following  questions  about  your 
organization  s  culture  for  the  work  location  you  designated  above. 

For  the  following  questions,  please  check  the  box  the  best  represents  your  opinion  about  the  numbered  statements: 


Strongly 

Somewhat 

Neither  agree 

Somewhat 

Strongly 

disagree., 

disagree^ 

nor  disagree.3 

agrees 

agree.5 

6.  People  I  work  with  are  direct  and  honest 
with  each  other. 

□ 

□ 

□ 

o 

o 

7.  People  I  work  with  accept  criticism 

77”© 

□  7  : 

7--  7  <3.7'77 

□ 

without  becoming  defensive. 

8.  People  I  work  with  resolve  disagreements 

□ 

□ 

□ 

□ 

□ 

cooperatively. 

9.  People  I  work  with  function  as  a  team. 

□ 

□ 

□ 

1 0.  People  I  work  with  are  cooperative  and 
considerate. 

□ 

□ 

□ 

□ 

o 

11.  People  I  work  with  constructively 

Q  7 

Q 

□ 

'  ri  ' 

2'.v  •  .■ 

□ 

confront  problems. 

O 

■  ShJ 

12.  People  I  work  with  are  good  listeners. 

□ 

□ 

o 

O 

13.  People  I  work  with  are  concerned  about 
:>  each  other. 

□ 

□ 

□ 

777a777 

14.  Labor  and  management  have  a  productive 

□ 

n 

□ 

□ 

□ 

working  relationship. 

15.  This  organization  motivates  me  to  put  out 

:  □ 

.7a  77' 

•••/•  ^q;  .;7'L 

If  tiff 

:  7;7'o7- 

my  best  efforts. 

16.  This  organization  respects  its  workers. 

□ 

0 

7  77 7©  7  7’ 

0 

1 7.  This  organization  treats  people  in  a 

□ 

□ 

O'  ' 

77'©f|7 

:'-7©7 

consistent  and  fan  manner. 

18.  Working  here  feels  like  being  part  of  a 
family. 

□ 

□ 

o 

0 

o 

19.  There  is  an  atmosphere  of  trust  in  this 

Q 

7  □ 

o 

o 

:V'77o'^7. 

organization. 

20.  This  organization  motivates  people  to  be 

□ 

□ 

□ 

□ 

□ 

efficient  and  productive. 

Please  turn  to  the  next  page. 
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Strongly 

Somewhat 

Neither  agree 

Somewhat 

Strongly 

disagree.] 

disagree^ 

nor  disagree^ 

agrees 

agree.s 

21.  I  get  enough  information  to  understand 

□ 

□ 

□ 

□ 

□ 

the  big  picture  here. 

22.  When  changes  are  made  the  reasons  why 

□ 

□ 

□ 

■  □ 

□ 

aireLniaidfe^ear:',;^  :y. 

23.  I  know  what's  happening  in  work  areas 

□ 

□ 

□ 

□ 

□ 

outside  of  my  own. 

24.  I  get  the  information  Ineed  to  do  my  job 

□ 

□ 

□ 

'  □  : 

□ 

'  Well.  IB'."  V’,  !■ ‘  1  vf 

25.  I  have  a  say  in  decisions  that  affect  my 

□ 

□ 

□ 

□ 

□ 

work. 

26;  I  am  ^ked  to  make  suggestions  about 

Vllffci  1 ;  f 

;f  ;af.|;  I  ,;; 

□ 

howto  do  my  job  better.  ; 

27.  This  organization  values  the  ideas  of 
workers  at  every  level. 

□ 

□ 

□ 

□ 

□ 

28.  My  opinions  count  in  Siis  organization. 

llliil 

□ 

□' 

llf-f  V.LJ:-  /;-l^ 

:  □ 

29.  My  job  requirements  are  made  clear  by 

□ 

□ 

□ 

□ 

□ 

my  supervisor.  _ _ 

30.  My ;  supearvisdr  teDs  n^^^  do  a  good 

1 1!  SI 

□  |l 

□ 

□ 

.•  ;  .  ,  Jy V  .  .  _  ' X;.  '  :• 1;.' • 

3 1 .  My  supervisor  takes  criticism  well. 

□  ■ 

□ 

□ . 

32  Miy  supervisor  delegates  responsibility. 

■  Off;  -3 

□' 

ifSil 

34.  My  supervisor  is  approachable. 

□ 

□ 

□ 

□ 

□ 

34.  My  supervisor  gives  me  cridcism  in  a 

□ 

□ 

□ 

□ 

\  positive maon^ri  : 

5./0.-U 

35.  My  supervisor  is  a  good  listener. 

□ 

□  1 

□ 

□ 

36.  My  supervisor  tells  me  how  Pm  doing. 

f  iffor  f  f 

,f 

13iiB 

37.  Decisions  made  at  meetings  get  put  into 
action. 

□ 

□ 

□ 

□ 

□ 

38.  Everyone  takes  part  in  discussions  at 

□  ; 

□ 

.  □ 

□ 

meetings.  • :  /  ; .  v .  ‘  ’ 1 ; 

□ 

■  ;s-;' 

□ 

39.  Our  discussions  in  meetings  stay  on 
track. 

□ 

□  - 

40.  Time  in  meetings  is  time  well  spent. 

.v ;  /cr 

41 .  Meetings  tap  the  creative  potential  of  the 
people  present. 

□ 

□ 

□ 

□ 

Please  turn  to  the  next  page. 
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Organizational  Communication 


This  is  a  series  of  questions  about  how  people  communicate  at  work.  Imagine  a  typical  week  at  work,  and  answer  the  questions 
accordingly.  Please  attempt  to  answer  all  the  questions. 

42.  Do  you  have  subordinates  working  for  you? 

G  Yes.j 
□  No.2 

Please  indicate  the  degree  of  your  agreement  with  each  statement  by  checking  the  most  appropriate  alternative  below  each 
statement. 

43.  How  free  do  you  feel  to  discuss  with  your  immediate  superior  the  problems  and  difficulties  you  have  in  your  job  without 
jeopardizing  your  position  or  having  it  "held  against  you"  later  in  this  organizations? 

Very  cautious.! 

G  Somewhat  cautious.2 
i:  Gjt  depends.  3 
G  Somewhat  free.4 
G  Completely  free_5 

50.  How  often  is  your  immediate  superior  successful  in  overcoming  restrictions  (such  as  regulations  or  quotas)  in  getting  you 
the  things  you  need  in  your  job  such  as  equipment,  personnel,  etc.? 

G  Never  successful.! 

Q  Very  seldom  successful^ 

G  Successful  half  of  the  time_3 
G  Successful  most  of  the  time.4 
G  Always  successful.5 

5 1 .  Immediate  superiors  at  times  must  make  decisions  that  seem  to  be  against  the  interests  of  their  subordinates.  When  this 
happens  to  you  as  a  subordinate,  how  much  trust  do  you  have  that  your  immediate  superior’s  decision  was  justified  by  other 
considerations? 

G  No  trust.! 

G  A  little  trusty 
•  Gi  Trust  half  of  the  time.3 
G  Trust  most  of  the  time.4 
.  ;G  Trust  all  of  the  time.5 

52.  In  general,  how  much  do  you  feel  that  your  immediate  superior  can  do  to  further  your  career  in  this  organization? 

G  Can  do  nothing.! 

G  Can  do  a  little_2 
G  Can  do  a  lot.3 

53.  How  much  weight  would  your  immediate  superior's  recommendation  have  in  any  decision  that  would  affect  your  standing  in 
this  organization,  such  as  promotions,  transfers,  etc.? 

Q  No  weight! 

Q  A  little  weighty 
G  A  lot  of  weighty 


Please  turn  to  the  next  page . 
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54.  As  part  of  your  present  job  plans,  how  much  do  you  want  a  promotion  to  a  higher  position  at  some  point  in  the  future? 
G  Content  as  I  am_i 

G  Somewhat  want  a  promotion^  , 

G  Very  much  want  a  promotion^ 

55.  How  important  is  it  for  you  to  progress  upward? 

Q  Not  at  all  important i  ;; ; 

G  Somewhat  important^ 

'  G  VeryimportanL3  fS'  :-i 

56.  To  what  extent  do  you  have  confidence  and  trust  in  your  immediate  superior  regarding  his/her  general  fairness? 

G  No  confident  and  trust.] 

G  A  littie  confident  and  trust2  j  7 j  % 

G  Confident  and  trust  half  of  the  time.3 
j  G  Confic^  of  the  tinted 

G  Complete  confident  and  trust.5 

While  working,  what  percentage  of  the  time  do  you  spend  interacting  with: 

'  •_  57.  %  . .  Jnafa^ 

58.  _ %  Subordinates? 

59.  _ %  Peers  (others  at  same  job  level)?  V  _ 

100% 

Of  the  total  time  you  engage  in  communications  while  on  the  job,  about  what  percentage  of  the  time  do  you  use  the  following 
methods  to  communicate: 


60. 

_ % 

Written? 

61. 

% 

Face-to-face? 

..  j:":.' ■'  , 

62. 

_ % 

Telephone? 

63. 

% 

Other?  (specify) 

;7  - 

100% 


How  accurate  is  information  that  you  receive  from: 


64.  Immediate  superiors? 

Never 

accurate.] 

Inaccurate  most 
of  the  time.2 

Accurate  half  of 
the  time.3 

□ 

Accurate  most 
of  the  time_4 

Always 

accurate.5 

mmmm: 

65.  Subordinates? 

□ 

□ 

□ 

□ 

□ 

66.  Peers  (others  at  your  job  level )? 

-O'"* 

:-M  IS* 

mmMmM 

How  clear  and  understandable  is  information  that  you  receive  from: 

Never 
clear,  i 

67.  Immediate  superiors?  Q-, 

Unclear  most  of 
the  time.2 

Clear  half  of  the 
time.3 

□ 

Clear  most  of 
the  time^ 

Always  clear. 

68.  Subordinates? 

□ 

□ 

□ 

□ 

69.  Peers  (others  at  your  job  level )? 

Please  turn  to  the  next  page. 
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70.  If  information  you  receive  is  not  clear  or  understandable,  do  you  generally  ask  for  help  or  clarification? 

□  Yes., 

ONo.2 

If  you  answered,  “No”,  please  proceed  to  question  74. 

Please  list  the  sources  you  generally  approach  for  clarification: 

Immediate  Superiors? 

Subordinates? 

Peers  (others  at  same  job  level)? 


74.  How  often  do  you  find  that  the  lack  of  available  information  hinders  your  performance  in  this  organization? 
Q  Never  hinders.! 

Q  Seldom hindCTS.2  • 

O  Hinders  half  of  the  time.3 

□  Hinders  most  of  die  time^ 

Q  Always  hinders_5 

75.  How  often  do  you  feel  that  you  receive  more  information  than  you  can  efficiently  use  in  this  organization? 

□  Never  have  too  much  information.! 

□  Seldom  have  too  much  infqrniation_2 

G  Have  too  much  information  half  of  the  time.3 
O  Have  too  much  information  most  of  the  time.4 
G  Always  have  too  much  information.5 

Of  the  total  time  you  spend  receiving  information  at  work,  what  percentage  comes  from: 

76.  _ _ %  Immediate  Superiors? 

77.  _ %  Subordinates? 

78  .  _ %  Peers  (others  at  same  job  level)? 

100% 

Of  the  total  time  you  spend  sending  information  at  work,  what  percentage  goes  to: 


79 . 

_ _% 

Immediate  Superiors? 

80. 

% 

Subordinates? 

81. 

_ % 

Peers  (others  at  same  job  level)? 

100% 


Please  turn  to  the  next  page . 
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72.  _ % 

73.  % 
100% 
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When  transmitting  information  to  others  in  this  organization,  do  you  summarize  by  emphasizing  those  aspects  that  are  important 
and  minimizing  those  aspects  that  are  unimportant?  How  often  do  you  do  this  with: 


82.  Immediate  superiors? 

83.  Subordinates? 

84.  Peers  (others  at  your  job  level  )? 


Never.i  Seldom.2 

rmv 

□  □ 

□  □  ■ 


Half  of  the  Most  of 

time.3  the  time_4 

l  □  □ 

. □ .  □ 

wdmwm  wmm 


Always.5 

□ 

rmw. 


Of  the  total  amount  of  information  you  receive  at  work,  how  much  do  you  pass  on  to: 


None.! 

Some.2  Half.3  Most* 

All., 

85.  Immediate  superiors? 

□ 

□  □  □ 

□ 

'  86:  Subordinate? 

□  □  •  "  •  □ 

mWMi 

87.  Peers  (others  at  your  job  level )? 

□ 

□ .  □ .  □ 

0 

How  desirable  do  you  feel  it  is  in  your  department  to  interact  frequently  with: 

Very 

Somewhat  Neither  desirable  or 

Somewhat 

Very 

undesirable.! 

undesirable.2  undesirable.3 

desirable^ 

desirable.^ 

88.  Immediate  superiors? 

■  □  □  ' 

: .vTO- 

89.  Subordinates? 

□ 

□  " .  □ 

□ 

□ 

90.  Peers  (others  at  your  job  level )?  4 

□  □' 

mmmm 

91 .  Are  there  forces  that  cause  you  to  distort  information  that  you  send  upward  in  this  organization?  Please  check  one . 

□  There  are  no  forces.! 

□  There  are  very  few  forces.2  k 

□  There  are  lots  of  forces.3 

92.  Put  check  the  box  next  to  the  expression  that  best  represent  how  you  feel  about  communications  in  general,  including  the 
amount  of  information  you  receive,  interaction  with  your  immediate  superior  and  others,  the  accuracy  of  information 

available,  etc.  Please  check  one. 

# 

□  Very  positive.! 

□  Positive^ 

G  Neutral.3 
G  Negative^ 

□  Verynegative.5 


Please  turn  to  the  next  page . 
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Computing  Issues 

Many  of  the  changes  to  the  business  processes  within  your  organization  will  require  you  to  use  computers  more  as 
part  of  your  daily  work.  Much  of  this  work  involves  communication  of  one  form  or  another.  Please  answer  the 
following  questions  about  your  computer  usage. 

93.  Do  you  have  access  to  a  computer? 

G  Yes,  I  have  one  for  my  personal  use.j 
O  Yes,  I  share  one  with  one  or  more  co-workers_2 
□  No.3 

If  you  answered  “No”,  please  proceed  to  the  next  page. 

94.  What  percentage  of  your  workday  would  you  estimate  that  you  spend  using  your  computer?  _ % 


95 .  Does  this  computer  provide  access  to  the  Internet? 

□  Yes., 

□  No_2 

96.  Do  you  have  your  own  electronic  mail  address  at  work? 

G  Yes.] 

ONo.2 


In  the  next  few  questions  we  would  like  you  to  assess  your  skill  at  using  different  types  of  computer  applications  as  part  of  your 
day-to-day  work: 

•  Applications  programs  like  Microsoft  Word,  Microsoft  Excel,  Microsoft  PowerPoint,  etc. 

•  Electronic  mail  programs  like  Microsoft  Outlook,  Eudora,  cc:  Mail,  etc. 

•  Internet  browsers  like  Netscape,  Internet  Explorer,  etc. 

•  Legacy  systems  like  PDMSS,  D035A/C/K,  EPA,  G03  7E/F,  etc. 

If  you  do  not  normally  use  that  type  of  application  at  work,  please  check  “Do  Not  Use 


Expert.! 

Intermediate^ 

Average.3 

Novice^ 

Do  Not 
Use.5 

97.  Applications  program 

□ 

□ 

□ 

□ 

□ 

98.  Electronic  mail  programs 

□ 

□ 

T 

99.  Internet  browsers 

□ 

□ 

□ 

□ 

□ 

100.  Legacy  systems’ 

:  □ 

□ 

For  these  same  types  of  computer  applications  we  *d  like  you  to  assess  the  level  of  training  you  have  received  to  date: 


101.  Applications  program 

More  than 
enough.! 

□ 

Enoughs 

□ 

Not  enough.3 

□ 

None_4 

□ 

Do  Not 
Use.5 

□ 

102.  Electronic  mail  programs 

-  □  - 

□  ■ 

: :: 

103.  Internet  browsers 

□ 

□ 

□ 

□ 

□ 

104.  Legacy  systems 

□ 

.  □ 

Please  turn  to  the  next  page. 
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Weapons  System  Support  Center  (WSSC) 

The  following  questions  are  intended  to  gauge  your  understanding  of  the  principles  of  the  Aircraft  Repair 
Enhancement  Program  (AREP)  and  the  operations  of  the  Weapons  System  Support  Center  (WSSC).  New  policies, 
procedures,  and  processes  are  being  put  into  place.  We  are  interested  in  your  understanding  of  how  things  should 
work  in  the  new  AREP  process ,  not  how  they  may  still  work  now. 


Please  complete  all  items. 

WSSC 

The  goal  of  AREP  is:  Please  check  all  that  apply 

L  To  privatize  the  supply  si^port  fbnction  of  AircraftPDM. 

2.  Solely  to  improve  the  flow  of  materiel  to  theaircxraft  utilizing  the  WSSC.  ^ 

3.  Tofiilly  synchronizenmterie^  facilitiestotfaeaircraft  per  theoperation- 


“Aircraft  management”  implements  an  automated,  tail-number  specific,  aircraft  production,  operation  level  schedule. 
This  information  is  made  available  to:  Please  check  all  that  apply. 


The  industrial  back  shops.  " 

■;v:  p: 

HQ  AFMC. 

Management  only  1  \  , 

□ 

All  support  functions  in  the  WSSC. 

□ 

wr-alore.  ; ;  ‘ -  3  •  •  vv  ■*  - 

The  following  functions  are  provided  by  the  WSSC:  Please  check  all  that  apply. 


9.  Workload  supportability  Q 

10.  Materiel  requisition  Q 

11.  Materiel  storage  and  distribution  ;  i;  .  -■r-Q  ;  r; 

12.  Materiel  management  □ 

13.  Procedures  and  analysis  Q 


The  logistics  support  variables  (materiel,  tools,  special  equipment,  HAZMAT,  personal  protective  clothing,  routed 
and  ROM  items)  required  at  the  tail  of  the  aircraft  will  be  managed  by:  Please  check  all  that  apply. 


14.  WSSC  Chief. 

15.  The  aircraft  synchronization  manager. 

16.  An  engineer. 

17.  The  aircraft  manager. 

18.  An  Airqcadft  Logistics  Specialist  (ALS). 


Please  turn  the  page. 


METRICS 


Both  aircraft  labor  and  logistic  support  staff  will  be  measured  by  their  ability  to  accomplish  tasks  in  accordance  with 
the  timing  prescribed  by:  Please  check  all  that  apply . 

19.  1st  line  manager. 

20.  The  Fixer. 

21 .  The  aircraft  production  schedule. 

22.  The  customer. 


Delays  to  schedule  will  be  a  major  measurement  for  both  the  Aircraft  Repair  Enhancement  Program  (AREP)  and 
DREP  which,  when  tracked  to  its  sources,  will  allow  for:  Please  check  all  that  apply. 

\  23 .  Continuous  improvement  in  schedule  acccnnpiishment.  Q 

24.  Identification  of  the  offending  staff.  □ 

WSSC  Performance  Metrics  include:  Please  check  all  that  apply. 

25  .  Mechanic  delays  due  to  parts,  tools,  engineering  and  CANNs 

26.  Unsupportable  AFMC  173  work  cards  issued  to  the  mechanic 

27.  Ui^redictable  repairs  scheduled  without  time  or  funds 

28.  Bill  of  Materiel  accuracy 

29.  100%  complete  operational  kits 

30.  Planned  vs.  unplanned  parts 

31.  PSI  Turnover  (Y  and  Pseudo  MIC) 

Customer  Metrics  include  all  of  the  following:  Please  check  all  that  apply. 

;  32.  Completion  of  aircraft  on  time  IAW  promised  delivery  date  '■■-■’.’'i 

33.  Cost  of  PDM/UDLM/MOD  □ 

34.  Quality  of  repair  ’  □ 

35.  Increased  aircraft  available  for  missions.  □ 


For  the  following  questions ,  please  check  “True”  if  you  believe  that  the  statement  applies  to  how  the  new  AREP  and 
WSSC  processes  are  supposed  to  work ,  not  how  they  might  work  now.  Please  complete  all  items. 

36.  HQ  has  approved  the  new  AREP/WSSC  related  Position  Descriptions.  True.i  □  False_2  O 

37.  The  yearly  reviews  of  persons  in  the  WSSC  have  changed  to  fit  the  new  positions.  True.]  O  False_2  Q 


□ 

□ 

□ 

□ 


Please  turn  the  page. 
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PROCESS 


38.  Under  the  WSSC,  the  ALS,  not  the  first  line  supervisor,  manages  people  on  the  aircraft.  True.]  □  False.2 

□ 

39.  The  tail  team  comprised  of  the  A/C  Production  First  Line,  ALS,  Planner,  and  FLS,  meet  regularly  to  discuss 
each  A/C's  status.  True.i  □  False.2  Q 

40.  Tail  team  meetings  are  only  held  when  the  aircraft  is  behind  schedule.  True.!  Q  False.2  Q 

41.  Clean  Bills  of  Material  (BOMs)  for  supportability  activities,  from  requirement  birth  in  the  System  Planning 
Office  (SPO),  until  variables  are  made  available  at  the  aircraft  are  a  must.  To  this  end,  planning  and  supply 
activities  must  be  highly  integrated  “from  SPO  to  Go”.  True.t  G  False.2  G 

42.  The  Forward  Support  Area  (FSA)  is  a  geographic  extension  of  the  WSSC  to  a  hangar  or  ramp,  where  the  aircraft 
are  being  repaired.  It  is  the  single  interface  point  to  the  mechanic  for  all  services  and  materiel,  parts,  special 
equipment,  special  tools,  and  hazardous  materiel,  which  flow  to  the  aircraft.  True.i  G  False.2  G 

43.  The  central  system  for  AREP  will  be  that  in  which  the  Production  Schedule  is  planned  and  executed... AREP 
currently  employs  the  Programmed  Depot  Maintenance  Scheduling  System  (PDMSS,  DSD  G097)  as  it’s 
"central"  information  system  True.i  G  False.2  G 

44.  The  heart  of  the  scheduling  system  is  the  Critical  Path  Network.  The  network  is  the  work  plan  by 
operation/task-level  for  the  project  (i.e.,  aircraft  tail  number).  All  resources  (parts,  tools,  skills,  equipment,  etc.) 
are  tied  to  the  network. 

True.)  G  False.2  Q 

45.  Once  a  critical  path  network  is  built  for  a  specific  aircraft  it  never  changes.  True.i  G  False.2  G 

46.  It  does  not  really  matter  when  the  materiel  arrives  at  the  aircraft,  as  long  as  it  gets  there  in  time  to  be  installed 
before  Functional  Test.  True.i  G  False.2  G 

SYNC  TEAM 

47.  A  Synchronization  Team  ("sync  team”)  will  be  made  up  of  the  competing  customer  representatives  and  the 
back-shop  SSC  to  work  the  "hot”  aircraft  priorities  from  a  center  perspective  when  conflicting  demands  for 
back-shop  capacity  exist.  True.i  G  Falser  Q 

48.  The  Sync  team’s  focus  is  on  the  aircraft  tail  number  schedule,  the  required  delivery  date  (RDD)  of  a  given 
operation,  and  the  back-shops’ expected  delivery  date  (EDD).  True.!  □  False_2  Q 

49.  When  there  are  several  customers  with  hot  aircraft  competing  for  backshop  services,  the  priority  will  go  to 
whoever  has  the  most  seniority.  True.i  Q  False.z  O 


Thank  you  for  your  time  and  effort 
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Communication  Baseline  Survey  Results: 
F-15  Weapon  System  Support  Center  (WSSC) 


ERIM 

Center  for  Electronic  Commerce 
28  June,  1999 


The  Center  for  Electronic  Commerce  recently  administered  a  survey  to  members  of  the  F-15  WSSC.  The  survey 
consisted  of  questions  about  culture,  communication,  computing  issues,  and  the  implementation  of  the  WSSC 
structure  and  processes.  The  data  collected  will  serve  to  provide  a  baseline  about  current  knowledge  and 
communications  practices  as  part  of  a  pilot  program  funded  by  the  Air  Force  Research  Laboratory  (AFRL/HESS) 
and  WR-ALC/RE.  W e  would  like  to  thank  the  WSSC  chief  and  our  respondents  for  all  of  their  time  and  effort. 


Interpreting  the  Results 


After  a  section  with  “General  Comments”,  the  sections  of  this  summary  match  the  section  headings  of  the  baselining 
questionnaire.  In  some  cases  questions  may  seem  to  apply  to  more  than  one  heading.  For  example,  there  are  some 
questions  under  the  heading  of  “Organizational  Communication”  that  are  related  to  culture.  For  the  purposes  of  this 
summaiy,  the  analysis  of  all  questions  will  occur  under  the  headings  in  the  original  survey. 

We  revised  a  copy  of  the  original  suivey  to  report  the  summary  statistics.  The  number  value  assigned  to  each  answer 
is  shown  in  small  type  next  to  the  question.  For  example,  in  question  6,  someone  checking  the  box  showing  that  they 
“somewhat  agree”  would  be  assigned  the  value  “4”  for  that  answer.  (This  is  NOT  included  in  this  Appendix.) 

The  summary  statistics  for  each  question  are  shown  next  to  the  question.  The  letter  N  represents  the  number  of 
usable  responses  we  received  for  that  item.  Next  to  that  number  we  show  the  mean  (or  arithmetic  average)  of  the 
responses  to  that  question.  In  some  cases,  several  items  are  also  combined  to  create  a  score  for  some  attribute  of 
your  organization,  such  teamwork  and  information  flow. 

For  most  of  the  “Yes/No”  questions  we  included  the  actual  number  of  responses,  i.e.,  if  21  people  answered  a 
question  Yes  ,  the  number  “21”  is  listed  next  to  that  response.  In  the  “Computing  Issues”  section  we  listed  the 
actual  number  of  responses  to  questions  about  computer  usage  and  training  (items  91  -  98). 

The  “Weapon  System  Support  Center”  section  was  intended  to  test  the  knowledge  of  the  WSSC  staff  about  the  new 
WSSC  structure,  policies,  and  processes.  As  such,  these  questions  included  both  true  and  false  statements.  In  the 
copy  of  the  survey  where  the  data  are  presented,  the  TRUE  statements  are  shown  in  bold. 

It  is  important  to  note  that  these  summary  statements  might  also  reflect  our  project  team’s  on-site  experience  at  WR- 
ALC  and  your  directorate.  In  many  cases,  the  survey  analyses  served  to  reinforce  the  impressions  we  developed 
during  our  considerable  time  on-site. 

General  Comments 

WSSC  management  distributed  80  surveys  to  staff  members.  Each  respondent  received  a  cover  letter  explaining  the 
purpose  of  the  survey,  a  blank  copy  of  the  questionnaire,  and  a  postage-paid  envelope  addressed  to  the  CEC. 

We  received  25  usable  responses,  for  a  response  rate  of  31%.  This  is  a  reasonable  response  rate  for  a  survey  of  this 
type.  It  is  important  to  note  that  those  who  choose  to  answer  the  questionnaire  may  differ  significantly  from  those 
who  did  not,  which  could  bias  the  results.  In  the  future,  special  efforts  to  increase  the  response  rate  of  such  surveys 
should  be  taken  to  ensure  that  the  results  are  indeed  representative  of  the  WSSC  population. 
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Most  of  our  rating  scales  included  5  options  with  the  middle  option  (3)  being  either  “Neither  agree  nor  disagree  , 
“Neither  desirable  or  undesirable”,  or  “Half  of  the  time”.  For  many  of  these  questions,  “3”  was  the  most  common 
choice  While  this  could  mean  that  the  respondent  indeed  had  no  real  opinion,  the  middle  choice  is  often  chosen 
routinely  as  a  way  to  quickly  complete  a  survey.  This  could  indicate  either  apathy  (they  didn’t  care  or  just  marked 
the  center  for  speed)  or  fear  of  repercussions.  As  we  stated  in  the  cover  letter,  all  responses  will  continue  to  be 
confidential,  with  only  summary  data  provided. 

Background 

We  clearly  did  not  phrase  Questions  2  and  4  correctly.  Since  the  WSSC  is  a  new  organization,  rmny  staff  members 
are  in  new  positions,  with  those  jobs  having  been  created  in  the  last  few  years.  The  data  do  clearly  show  that  the 
respondents  are  mostly  non-union  employees. 


Culture 

The  “Culture”  section  of  this  survey  was  derived  from  an  existing  questionnaire.  In  that  questionnaire,  the  groups  of 
questions  in  this  section  represented  different  higher  level  concepts. 


•  Questions  6  - 13  -  Teamwork 

•  Questions  14  -  20  -  Morale 

•  Questions  21  -  24  -  Information  Flow 

•  Questions  25  -  28  -  Involvement 

•  Questions  29  -  36  -  Supervision 

•  Questions  37  -  41  -  Meetings 


The  ratings  scales  are  from  1-5,  with  1  being  the  low  score.  With  this  system,  higher  scores  are  better  both  on 
individual  questions  and  each  concept.  While  the  concept  scores  are  important,  the  answers  for  die  individual  items 
can  help  indicate  specific  areas  for  improvement.  For  example,  answers  for  questions  37  through  41  can  pinpoint 
how  your  staff  believes  meetings  can  best  be  improved. 


We’ll  report  on  each  concept  in  turn. 

Teamwork:  The  items  ask  about  honesty,  problem  resolution,  and  team  interactions.  A  score  of  3.2  is  low  here. 
The  lowest  item,  ability  to  accept  criticism  without  becoming  defensive  (2.6),  is  an  area  of  concern. 


Morale:  These  questions  focus  on  working  relationships,  motivation,  trust,  and  respect.  Morale  is  low  (2.8),  with 
four  out  of  the  six  scale  items  scoring  below  3.  Motivation  and  respect  are  the  only  items  above  . 


Information  Flow:  The  questions  on  information  flow  are  general  in  this  section,  asking  about  quantity,  quality,  and 
accessibility  of  information.  (Information  flow  is  also  considered  in  more  detail  m  the  Organizational 
Communication  section.)  While  most  of  the  ratings  were  middle  of  the  road,  people  clearly  do  not  feel  as  if  they 
understand  why  specific  processes  are  being  implemented  as  the  solution  (2.2  on  item  22). 


Involvement:  This  concept  includes  both  whether  people  can  participate  and  if  their  actions  are  valued.  While 
people  say  they  are  asked  for  their  input  (items  25  and  26),  they  feel  that  their  input  does  not  count  (items  27  and 

28). 

Supervision:  This  concept  focuses  on  aspects  for  the  respondents’  immediate  supervisor.  This  is  the  highest  rated 
concept  (3 .5),  with  the  only  reservations  being  that  workers  would  like  work  requirements  made  more  clear  (item  ) 
and  that  more  feedback  is  desired  (30,  36). 


Meetings:  Questions  here  ask  about  attributes  of  successful  meetings  at  any  organization  such  as  meetings  stay  on 
track,  everyone  participates,  and  decisions  made  at  the  meeting  are  enacted.  This  was  the  lowest  scoring  concept 
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(2.6),  with  all  but  one  item  below  3.  Meeting  time  is  not  seen  as  being  well  spent  and  decisions  that  are  made  are  not 
put  into  action. 

Organizational  Communication 

This  section  of  the  questionnaire  was  also  based  on  an  existing  survey.  While  these  items  are  not  grouped  into 
“concepts”  as  in  the  “Culture”  section,  there  are  some  natural  groupings.  For  example,  items  43-50  focus  on  the 
respondent’s  relationship  with  their  immediate  supervisor.  Specific  concerns  include  retribution,  advancement,  and 
trust. 

Employees  indicated  that  they  do  not  always  feel  free  to  discuss  problems  with  their  supervisor  (item  43).  These 
same  supervisors  bat  about  .500  when  trying  to  help  you  get  what  you  need  to  do  your  job  (item  44)  and  with  your 
trust  in  their  decisions  (item  45).  People  say  they  want  to  get  promoted  (items  48  and  49),  but  feel  that  their 
supervisors  can’t  really  help  (items  46  and  47). 

It  is  most  desirable  for  you  to  interact  with  your  peers,  less  so  with  your  subordinates  and  superiors  (items  82-84). 
You  spend  most  of  your  communication  time  interacting  with  your  peers  (68%  in  item  53.)  You  get  5 1%  of  your 
information  from  peers,  with  superiors  a  distant  second  at  30%  (items  70-72).  Your  communication  at  work  is 
generally  either  face-to-face  or  on  the  phone  (items  54-57).  There  is  some  email  use,  but  it  is  not  really  high 
considering  how  much  you  use  your  computers  (see  the  next  section).  The  information  you  get  from  both  above  and 
below  your  level  is  accurate  and  understandable  much  of  the  time  (items  58-63).  If  you  have  a  question  you  ask  it,  of 
peers  46%  of  the  time  and  superiors  42%  (items  64-67).  While  you  like  what  you  get,  you  may  not  be  getting 
enough  of  what  you  need.  The  score  on  item  68  indicates  that  you  are  hindered  in  your  job  nearly  half  of  the  time  by 
lack  of  information.  In  item  69  you  said  you  seldom  have  too  much  information. 

When  you  are  distributing  information  you  tend  to  filter  what  you  send  (items  76-78)  and  send  more  to  your  peers 
than  to  your  supervisors  or  subordinates.  Item  85  asked  about  whether  you  distort  information,  certainly  a  much 
stronger  word  than  what  we  used  in  items  76  to  78  (summarizing  important  aspects  and  minimizing  unimportant 
ones).  And,  when  we  put  it  this  way,  you  said  there  are  few  forces  that  might  cause  you  to  distort  information  you 
send  to  your  superiors. 

Overall,  you  rated  your  communications  only  slightly  better  than  neutral  (item  86).  Since  much  of  what  you  do 
requires  effective  communication,  this  is  a  low  score. 

Computing  Issues 

WR-ALC  has  undergone  many  major  changes  over  the  last  few  years  and  all  of  the  contained  a  significant 
information  technology  component.  The  use  of  computers  will  become  increasingly  central  to  the  way  you  do 
business.  This  includes  as  modes  of  communication.  As  such,  we  thought  it  was  important  to  include  in  our 
baselining  an  overview  of  your  computer  access,  usage,  knowledge,  and  training. 

Most  of  the  respondents  have  a  computer  for  their  personal  use,  with  their  own  email  address  and  access  to  the 
Internet.  They  are  spending  more  than  half  their  workday,  63%,  at  their  machines.  Overall  skill  level  seems 
adequate,  but  most  of  you  believe  you  could  benefit  from  more  training. 

Weapon  System  Support  Center  (WSSC) 

Part  of  any  change  effort  is  building  an  understanding  of  the  new  structure,  policies,  and  processes.  The  questions  in 
this  section  were  constructed  by  combining: 

•  Facts  taken  from  policy  manuals,  briefings,  and  other  materials  with 

•  Popular  misconceptions  about  the  change  provided  by  WSSC  staff 
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We  decided  to  report  your  “scores”  in  several  different  ways.  AH  are  based  on  a  percentage,  as  on  a  test  in  school, 
from  0%  to  1 00%.  First  is  an  overall  score  for  the  “class”,  the  whole  WSSC. 


Overall  Results  Histogram 


Overall,  the  respondent’s  scored  84%,  an 
excellent  result.  To  see  what  aspects  of  the 
change  to  the  WSSC  structure  might  be 
more  troublesome,  we  broke  out  the  scores 
by  question  groups  on  the  survey. 


•  WSSC:  Questions  1-18 

•  Metrics:  19-37 

•  Process:  38-46 

•  Sync  Team:  47-49 


By  looking  at  these  groupings  you  can  get  an 
idea  about  the  aspects  of  the  WSSC  change 
on  which  staff  members  are  not  yet  clear. 


Questions  of  metrics  and  process  caused  the  respondents  the  most  trouble,  if  you  can  call  B’s  and  C’s  real  trouble. 
Based  on  these  results,  broad-based  knowledge  of  the  change  to  the  WSSC  structure  and  processes  is  not  a  problem 
for  the  F- 15  WSSC. 
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For  More  Information 


Please  contact:  Stanley  M.  Przybylinski 

ERIM 

Center  for  Electronic  Commerce 
2901  Hubbard  Road 
Ann  Arbor,  MI  48105 
Phone:  (734)  769-4517 
FAX:  (734)213-3405 
Email:  smp@erim.org 

Thank  you  again  for  the  time  and  effort  you  committed  to  this  survey. 
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Communication  Baseline  Survey  Results: 
C-141  Weapon  System  Support  Center  (WSSC) 

ERIM 

Center  for  Electronic  Commerce 
28  June,  1999 


The  Center  for  Electronic  Commerce  recently  administered  a  survey  to  members  of  the  C-141  WSSC.  The  survey 
consisted  of  questions  about  culture,  communication,  computing  issues,  and  the  implementation  of  the  WSSC 
structure  and  processes.  The  data  collected  will  serve  to  provide  a  baseline  about  current  knowledge  and 
communications  practices  as  part  of  a  pilot  program  funded  by  the  Air  Force  Research  Laboratory  (AFRL/HESS) 
and  WR-ALC/RE.  We  would  like  to  thank  the  WSSC  chief  and  our  respondents  for  all  of  their  time  and  effort. 

Interpreting  the  Results 


After  a  section  with  “General  Comments”,  the  sections  of  this  summary  match  the  section  headings  of  the  baselining 
questionnaire.  In  some  cases  questions  may  seem  to  apply  to  more  than  one  heading.  For  example,  there  are  some 
questions  under  the  heading  of  “Organizational  Communication”  that  are  related  to  culture.  For  the  purposes  of  this 
summary,  the  analysis  of  all  questions  will  occur  under  the  headings  in  the  original  survey. 

We  revised  a  copy  of  the  original  survey  to  report  the  summary  statistics.  The  number  value  assigned  to  each  answer 
is  shown  in  small  type  next  to  the  question.  For  example,  in  question  6,  someone  checking  the  box  showing  that  they 
“somewhat  agree”  would  be  assigned  the  value  “4”  for  that  answer.  (This  is  NOT  included  in  this  Appendix.) 

The  summary  statistics  for  each  question  are  shown  next  to  the  question.  The  letter  N  represents  the  number  of 
usable  responses  we  received  for  that  item.  Next  to  that  number  we  show  the  mean  (or  arithmetic  average)  of  the 
responses  to  that  question.  In  some  cases,  several  items  are  also  combined  to  create  a  score  for  some  attribute  of 
your  organization,  such  teamwork  and  information  flow. 

For  most  of  the  “Yes/No”  questions  we  included  the  actual  number  of  responses,  i.e.,  if  21  people  answered  a 
question  “Yes”,  the  number  “21”  is  listed  next  to  that  response.  In  the  “Computing  Issues”  section  we  listed  the 
actual  number  of  responses  to  questions  about  computer  usage  and  training  (items  91  -  98). 

The  “Weapon  System  Support  Center”  section  was  intended  to  test  the  knowledge  of  the  WSSC  staff  about  the  new 
WSSC  structure,  policies,  and  processes.  As  such,  these  questions  included  both  true  and  false  statements.  In  the 
copy  of  the  survey  where  the  data  are  presented,  the  TRUE  statements  are  shown  in  bold. 

It  is  important  to  note  that  these  summary  statements  might  also  reflect  our  project  team’s  on-site  experience  at  WR- 
ALC  and  your  directorate.  In  many  cases,  the  survey  analyses  served  to  reinforce  the  impressions  we  developed 
during  our  considerable  time  on-site. 

General  Comments 

WSSC  management  distributed  96  surveys  to  staff  members.  Each  respondent  received  a  cover  letter  explaining  the 
purpose  of  the  survey,  a  blank  copy  of  the  questionnaire,  and  a  postage-paid  envelope  addressed  to  the  CEC. 

We  received  29  usable  responses,  for  a  response  rate  of  30%.  This  is  a  reasonable  response  rate  for  a  survey  of  this 
type.  It  is  important  to  note  that  those  who  choose  to  answer  the  questionnaire  may  differ  significantly  from  those 
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who  did  not,  which  could  bias  the  results.  In  the  future,  special  efforts  to  increase  the  response  rate  of  such  surveys 
should  be  taken  to  ensure  that  the  results  are  indeed  representative  of  the  WSSC  population. 


Most  of  our  rating  scales  included  5  options  with  the  middle  option  (3)  being  either  “Neither  agree  nor  disagree  , 
“Neither  desirable  or  undesirable”,  or  “Half  of  the  time”.  For  many  of  these  questions  “3”  was  the  most  common 
choice  While  this  could  mean  that  the  respondent  indeed  had  no  real  opimon,  the  middle  choice  is  often  chosen 
routinely  as  a  way  to  quickly  complete  a  survey.  This  could  indicate  either  apathy  (they  didn  t  care  or  just  marked 
the  center  for  speed)  or  fear  of  repercussions.  As  we  stated  in  the  cover  letter,  all  responses  will  contmue  to  be 
confidential,  with  only  summary  data  provided. 


Background 


We  clearly  did  not  phrase  Questions  2  and  4  correctly.  Since  the  WSSC  is  a  new  organization,  many  staff  members 
are  in  new  positions,  with  those  jobs  having  been  created  in  the  last  few  years.  The  data  do  clearly  show  that  the 
respondents  are  mostly  non-union  employees. 


Culture 


The  “Culture”  section  of  this  survey  was  derived  from  an  existing  questionnaire.  In  that  questionnaire,  the  groups  of 
questions  in  this  section  represented  different  higher  level  concepts: 


•  Questions  6  -  13  -  Teamwork 

•  Questions  14  -  20  -  Morale 

•  Questions  21  -  24  -  Information  Flow 

•  Questions  25  -  28  -  Involvement 

•  Questions  29  -  36  -  Supervision 

•  Questions  37  -  4 1  -  Meetings 


The  ratings  scales  are  from  1-5,  with  1  being  the  low  score.  With  this  system,  higher  scores  are  better  both  on 
individual  questions  and  for  each  concept.  While  the  concept  scores  are  important,  the  answers  for  the  individual 
items  can  help  indicate  specific  areas  for  improvement.  For  example,  answers  for  questions  37  through  41  can 
pinpoint  how  your  staff  believes  meetings  can  best  be  improved. 


We’ll  report  on  each  concept  in  turn. 

Teamwork:  The  items  ask  about  honesty,  problem  resolution,  and  team  interactions.  A  score  of  2.9  is  very  low 
here.  The  lowest  item,  ability  to  accept  criticism  without  becoming  defensive  (2.2),  is  an  area  of  concern. 


Morale-  These  questions  focus  on  working  relationships,  motivation,  trust,  and  respect.  Morale  is  very  low  (2.1), 
with  all  of  the  six  scale  items  scoring  below  3.  Trust  is  particularly  low  (item  19-1.9  out  of  5). 


Information  Flow:  The  questions  on  information  flow  are  general  in  this  section,  asking  about  quantity,  quality,  and 
accessibility  of  information.  (Information  flow  is  also  considered  in  more  detail  m  the  Organizational 
Communication  section.)  Overall  information  flow  scored  a  2.4  out  of  5.  The  lowest  score  1.7,  reflected  the 
respondents  belief  that  that  the  reasons  that  specific  changes  are  being  made  are  not  made  clear. 

Involvement:  This  concept  includes  both  whether  people  can  participate  and  if  their ^actions ;  are  valued .  AjM* 
overall  score  is  below  3.  While  people  say  they  may  be  asked  for  their  input  (items  25  and  26),  they  feel  that  their 
input  does  not  count  (items  27  and  28). 

Supervision:  This  concept  focuses  on  aspects  for  the  respondents’  immediate  supervisor.  This  is  the  highest  rated 

concept  (3.6).  This  result  is  somewhat  confusing  given  the  low  scores  on  the  other  concepts.  People  could  ear 

retribution  as  a  result  of  a  low  score. 
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Meetings:  Questions  here  ask  about  attributes  of  successful  meetings  at  any  organization,  such  as  meetings  stay  on 
track,  everyone  participates,  and  decisions  made  at  the  meeting  are  enacted.  This  was  low  (2.6),  with  all  items  below 
3.  Meeting  time  is  not  seen  as  being  well  spent  and  decisions  that  are  made  are  not  put  into  action. 

Organizational  Communication 

This  section  of  the  questionnaire  was  also  based  on  an  existing  survey.  While  these  items  are  not  grouped  into 
“concepts”  as  in  the  “Culture”  section,  there  are  some  natural  groupings.  For  example,  items  43-50  focus  on  the 
respondent’s  relationship  with  their  immediate  supervisor.  Specific  concerns  include  retribution,  advancement,  and 
trust. 

Employees  indicated  that  they  are  cautious  about  discussing  problems  with  their  supervisors  (item  43).  These  same 
supervisors  are  seldom  successful  when  trying  to  help  you  get  what  you  need  to  do  your  job  (item  44)  and  you  have 
little  trust  in  their  decisions  (item  45).  People  are  middle  of  the  road  about  getting  promoted  (items  48  and  49),  but 
feel  that  their  supervisors  can’t  help  (items  46  and  47). 

It  is  most  desirable  for  you  to  interact  with  your  peers,  less  so  with  your  subordinates  and  superiors  (items  82-84). 
You  spend  most  of  your  communication  time  interacting  with  your  peers  (66%  in  item  53.)  You  get  57%  of  your 
information  from  peers,  with  superiors  a  distant  second  at  26%  (items  70-72).  Your  communication  at  work  is 
generally  either  face-to-face  (47%)  or  on  the  phone  (32)  (items  54-57).  There  is  some  email  use,  but  it  is  not  really 
high  considering  how  much  you  use  your  computers  (see  the  next  section).  The  information  you  get  from  both  above 
and  below  your  level  is  accurate  and  understandable  more  than  half  of  the  time  (items  58-63).  If  you  have  a  question 
almost  all  of  you  ask  it,  of  peers  52%  of  the  time  and  superiors  40%  (items  64-67).  While  you  like  what  you  get,  you 
may  not  be  getting  enough  of  what  you  need.  The  score  on  item  68  indicates  that  you  are  hindered  in  your  job  nearly 
half  of  the  time  by  lack  of  information.  In  item  69  you  said  you  seldom  have  too  much  information. 

When  you  distribute  information  you  tend  to  filter  what  you  send  more  than  half  the  time  (items  76-78)  and  send 
more  to  your  peers  and  subordinates  than  to  your  supervisors.  Item  85  asked  about  whether  you  distort  information, 
certainly  a  much  stronger  word  than  what  we  used  in  items  76  to  78  (summarizing  important  aspects  and  minimizing 
unimportant  ones).  And,  when  we  put  it  this  way,  you  said  there  are  few  forces  that  might  cause  you  to  distort 
information  you  send  to  your  superiors. 

Overall,  you  rated  your  communications  slightly  worse  than  neutral  (item  86).  Since  much  of  what  you  do  requires 
effective  communication,  this  is  a  low  score. 

Computing  Issues 

WR-ALC  has  undergone  many  major  changes  over  the  last  few  years  and  all  of  the  contained  a  significant 
information  technology  component.  The  use  of  computers  will  become  increasingly  central  to  the  way  you  do 
business.  This  includes  as  modes  of  communication.  As  such,  we  thought  it  was  important  to  include  in  our 
baselining  an  overview  of  your  computer  access,  usage,  knowledge,  and  training. 

Most  of  the  respondents  have  a  computer  for  their  personal  use,  with  their  own  email  address  and  access  to  the 
Internet.  They  are  spending  more  than  half  their  workday,  58%,  at  their  machine.  Overall  skill  level  seems 
adequate,  but  most  of  you  believe  you  could  benefit  from  more  training. 

Weapon  System  Support  Center  (WSSC) 

Part  of  any  change  effort  is  building  an  understanding  of  the  new  structure,  policies,  and  processes.  The  questions  in 
this  section  were  constructed  by  combining: 

•  Facts  taken  from  policy  manuals,  briefings,  and  other  materials  with 
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•  Popular  misconceptions  about  the  change  provided  by  WSSC  staff. 

We  decided  to  report  your  “scores”  in  several  different  ways.  All  are  based  on  a  percentage,  as  on  a  test  its  school 
from  0%  to  100%.  First  is  an  overall  score  for  the  “class”,  the  whole  WSSC. 

Overall,  the  respondent’s  scored  66%,  a  poor 
result.  About  half  the  group  scored  above  a  “D” 
and  half  below.  There  could  be  some  response 
bias  here.  Staff  with  low  morale  may  have  little 
motivation  to  become  knowledgeable  about  the 
changes. 

Then  we  also  break  out  the  scores  by  question 
groups  on  the  survey. 

•  WSSC:  Questions  1-18 

•  Metrics:  19-3? 

»  Process:  38-46 

•  Sync  Team:  47-49 

By  looking  at  these  groupings  you  can  get  an.  idea 
about  the  aspects  of  the  WSSC  change  on  which 
staff  members  are  not  yet  dear. 


Results  Histograms  by  Question  Area 
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Staff  members  had  more  problems  with  process  and  metrics  questions  than  WSSC  structure  and  the  Sync  Team. 
Since  process  and  measurement  issues  are  crucial  in  the  new  design,  lack  of  knowledge  of  these  areas  is  an  important 
issue. 

For  More  Information 

Please  contact:  Stanley  M.  Przybylinski 

ERIM 

Center  for  Electronic  Commerce 
2901  Hubbard  Road 
Ann  Arbor,  MI  48105 
Phone:  (734)  769-4517 
FAX:  (734)  213-3405 
Email:  smp@erim.org 

Thank  you  again  for  the  time  and  effort  you  committed  to  this  survey. 
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Appendix  F.  DRMS  Effectiveness  Tests 

This  appendix  includes  the  following  materials  from  our  communication  effectiveness  tests  at  DRMS: 

•  Survey  1  and  its  results.  Please  note  that  these  surveys  were  delivered  over  the  Web.  Copying  the  Web  files 
and  formatting  them  as  Word  documents  substantially  changed  their  appearance. 

•  Survey  2  and  its  results. 

•  Survey  3  and  its  results 
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CAP  Survey  Form  #1  -  Bidder  Registration  Process 


1 .  Before  receiving  this  questionnaire,  did  you  know  any  of  the  information  in  the  box  below? 
Please  check  each  of  the  items  you  knew. 


p  R1 

The  new  bidder  registration  process  will  help  DRMS  ensure  that  debarred  or  indebted  bidders  are 
prevented  from  participating  in  DRMS  sales. 

The  new  bidder  registration  process  will  help  DRMS  to  improve  accuracy. 

]□ 

The  new  bidder  process  will  help  reduce  duplicate  records  in  DNSP  and  CORALS. 

|o  R4 

The  CORALS  system  is  the  most  accurate  way  to  determine  whether  a  bidder  is  indebted  or  debarred. 

jP  R5 

Using  DAISY  and  DNSP  at  the  same  time  can  shorten  the  registration  time. 

2.  If  you  had  the  information  in  the  box  but  you  heard  it  differently,  please  write  the  facts  you  heard  below. 


If  you  checked  above  that  you  knew  any  of  the  information ,  please  complete  the  questionnaire  by  providing  the 
information  requested  below .  , 

If  you  did  not  check  any  of  the  items  (you  did  not  know  any  of  the  information  in  the  box),  you  have  completed  the 
questionnaire.  Please  submit  the  questionnaire  using  this  button: 


Thank  you  very  much  for  your  cooperation. 


3.  From  whom  did  you  first  receive  the  information  in  the  box? 
Please  check  only  one. 

'■O 

Superiors  above  your  immediate  superior? 

C  • 

Immediate  superiors? 

C 

Peers  (others  at  your  j  ob  level)? 

C 

Subordinates? 


4.  Where  were  you  when  you  first  received  the  information  in  the  box  above? 

Please  review  all  of  the  options  and  select  the  statement  that  best  represents  your  location.  Please  check 
only  one. 

C  I  was  working  within  my  designated  work  area  (at  my  desk,  board,  workstation,  etc.) 

O  I  was  working  within  my  department,  but  outside  of  my  designated  work  area. 

O  I  was  working  outside  of  my  department 

C  I  was  on  break  outside  of  my  department,  but  still  within  the  building  (lunch,  coffee  break,  etc.) 

C  I  was  out  of  the  building  and  not  working 
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5.  How  long  ago  did  you  first  receive  the  information  in  the  box? 
Check  the  approximate  time . 


^  Today 

m 

1  -  2  weeks  ago 

c 

1-2  days  ago 

b' 

VC,  3-4  weeks  ago 

r 

3-4  days  ago 

H  5-6  weeks  ago 

P'  5-6  days  ago 

6.  By  what  method  did  you  first  receive  the  information  in  the  box  above? 

Please  review  all  of  the  options  and  select  the  statement  that  best  represents  your  location.  Please  check 
only  one  of  the  following  methods. 

Written  or  visual  methods 

p 

Personal  letter 

SO 

Letter,  memo,  or  service  program 


(0 

p 

fep 

p 

p 

b 

c 

b 

O 

b 


Electronic  mail  message 
Facsimile  (FAX) 

Newsletter 
Magazine 
Videotape  message 
Company  film 

Public  newspaper  or  magazine 
Records 

Our  internal  world  wide  web  site 
An  external  world  wide  web  site 


Talking  or  sound  methods 

b 

Face-to-face  conversation 

C 

Talking  over  the  telephone 

b 

Talking  (and  listening)  in  a  small  group  of  two  or  more 

C* 

Attending  an  organized  meeting,  briefing  or  conference 

]£)  . 

C  Overhearing  what  someone  else  said 

b 

Radio  or  television 


Miscellaneous 

f) 

1  am  the  subject/source  of  the  message 

br¬ 

other,  (Please  explain.) 
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Thank  you  very  much  for  your  cooperation. 


DRMS  Communication  Effectiveness  Survey  #1 
Bidder  Registration  Process 

ERIM 

May  11, 1999 


In  early  April,  a  change  in  the  Bidder  registration  process  was  communicated  from  DRMS  HQ  to  the  Zone  Manager 
and  DRMO  Chiefs.  Since  this  policy  affected  bidding,  Distribution  Branch  Chiefs,  Sales  Contracting  Officers, 
Cashiers,  and  Marketing  Technicians  would  need  to  know  about  the  change.  The  original  message  requested  that  all 
affected  parties  review  the  details  of  the  change  available  on  the  DRMS  Internal  World  Wide  Web  site.  (While  we 
were  told  this  was  a  “change”,  it  was  more  of  a  clarification  of  desired  best  practice.) 

Respondents  for  this  survey  were  solicited  using  emails  sent  to  the  Zone  Managers  and  DRMO  Chiefs.  The  Zone 
Managers  were  provided  a  URL  to  a  password-protected  WWW  site.  They  were  provided  a  user  name,  password, 
and  code  number  to  identify  them  by  Zone.  Chief  were  sent  a  similar  email,  requesting  that  they  respond  and  to 
forward  a  similar  request  to  the  appropriate  staff  members.  This  indirect  contact  was  necessary  since  HQ  could  not 
provide  direct  email  contact  with  the  specific  staff  categories.  For  this  same  reason,  we  can  only  estimate  the 
possible  number  of  respondents.  Based  on  12  Zone  Managers  and  83  DRMOs,  we  estimate  that  there  could  be 
approximately  400  respondents. 

Results 

These  results  are  based  on  1 14  respondents.  Overall,  this  is  a  29%  response  rate.  This  is  reasonable  for  a  survey  of 
this  type.  Responses  rate  by  zone  vary  widely.  We  arbitrarily  numbered  the  12  Zones  from  1  to  12.  The  rates  for 
each  Zone  are  shown  below. 


Zone  1  Zone  2  Zone  3  Zone  4  Zones  Zone  6  Zone  7  Zone  8  Zone  9  Zone  10  Zone  11  Zone  12 
Rate  46  36  18  24  14  10  32  7  36  29  50  35 


Originally  we  wanted  to  track  the  response  down  the  chain  of  command,  to  see  how  well  each  message  traveled  from 
the  Zone  Manager  to  their  Chiefs,  then  from  each  Chief  to  their  first  line  supervisors,  and  finally  from  those 
supervisors  to  their  subordinates.  As  stated  above,  the  best  we  could  do  with  the  email  contact  information  available 
was  to  get  to  each  Chief  specifically  and  then  ask  them  to  pass  a  separate  message  to  their  first  lines  and  the 
appropriate  subordinates.  We  assigned  code  numbers  to  each  respondent  that  would  identify  their  position  in  the 
chain.  Unfortunately,  our  instructions  must  not  have  been  explicit  enough  because  many  Chiefs  passed  on  their 
message  (with  their  code  number)  to  their  subordinates.  Thus,  our  database  contains  data  apparently  from  multiple 
Chiefs  from  each  DRMO.  In  a  subsequent  survey,  we  tried  a  different  approach  to  avoid  this  problem. 

The  first  question  includes  five  factual  statements  about  the  Bidder  registration  process.  Ideally,  the  respondents 
should  know  all  five.  Scoring  "five  out  of  five"  would  yield  a  score  of  100%,  "four  out  of  five"  80%  and  so  on.  As 
can  be  seen  in  the  graph  below,  a  majority  of  respondents  did  not  know  about  the  changes  in  the  Bidder  registration 
process. 
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This  message  was  intended  to  be  sent  down  the  DRMS  hierarchy  from  HQ  to  the  Zone  Managers,  to  their  DRMQ 
Chiefs,  and  then  to  the  Distribution  Branch  Chiefs,  Sales  Contracting  Officers,  Cashiers,  and  Marketing  Technicians, 
Given  this,  you  would  expect  most  respondents  to  hear  the  message  from  superiors  or  peers.  This  is  reflected  in  the 
chart  below. 


Message  Source 


11% 


24% 


32% 


U  Superiors  Above 
■  Immediate  Superior 

□  Peers 

□  Subordinates 


33% 


Where  Received 


6%  2% 


The  original  message  from  HQ  about  the  new  Bidder  registration  process  was  sent  out  in  the  first  week  of  April. 

This  survey  was  conducted  during  the  last  week  of  April  and  first  week  of  May.  As  a  manager,  you  would  hope  that 
most  people  received  the  message  soon  after  you  sent  it,  i.e.,  "34  weeks  ago".  The  graph  below  shows  that  4 1%  did. 
It  is  not  clear  whether  the  29%  who  said  ’“5-6  weeks  ago"  just  lost  track  of  time 


When  Received 


5%  6% 


leToday 

*1-2 

weeks  ago: 

□  1-2 

days  ago  ; 

^□34 

weeks  ago  - 

:*34 

days  ago 

105-6 

weeks  ago 

:*5-6 

days  ago  ; 

This  process  change  was  communicated  via  email  to  the  Zone  Managers  and  DRMO  chiefs  using  email.  In  the 
message,  they  were  told  to  go  to  the  DRMS  Internal  WWW  site  to  get  more  details.  The  DRMO  Chiefs  could  then 
provide  the  information  to  their  staff  their  normal  channels.  As  you  can  see  in  the  chart  below'  most  DRMS  staff 
received  the  message  via  email. 
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Method 


13  Letter,  memo 
B  Email 

□  Newsletter 

□  Internal  WWW 
B  Face-to-face 

□  Telephone 
B  Small  Group 

□  Organized  meeting 
B  Other 


In  this  survey,  there  are  two  opportunities  for  respondents  to  provide  input  in  their  own  words,  question  2  (“did  you 
hear  the  information  in  question  differently?”)  and  question  6  (“By  what  other  means  did  you  first  receive  the 
information  in  question?”).  For  completeness,  we  include  those  comments  modified  only  for  spelling  and  to  remove 
comments  that  could  identify  specific  respondents. 

Comments  in  Question  2 

I  have  not  heard  of  a  new  registration  system 

R4-CORALS  is  not  the  most  accurate  system  since  it  does  not  append  with  DNSP... CORALS  is  the  system  at  each 
HP  and  is  dependent  on  what  the  cashier/sales  tech  inputs. 

"The  registration  time  using  DNSP  and  daisy  takes  a  long  time  and  should  have  been  incorporated  into  daisy  corals 
before  they  implemented  it.  It  is  very  hard  to  have  both  screens  open  at  the  same  time  as  your  computer  is 
overloaded  as  it  is  and  everything  makes  it  more  difficult  to  input  faster" 

"R3  comment:  we  all  need  to  get  into  the  same  sheet  of  music  when  it  comes  to  the  how  a  customer  is  to  be 
registered.  Validation  from  some  type  of  I.D.  for  legal  name,  customers  need  to  be  registered  with  one  standard 
pattern,  legal  last  name,  first.  Middle  name  or  initial  (with  the  dot  at  the  end  or  not).  We  come  to  the  problem  that 
the  customer  registered  one  way  at  one  site  and  is  loaded  with  bin  number  and  then  another  site  registers  the 
customer  another  way.  Example,  Black,  Thomas  p.  and  Black,  Tommy  or  Black,  Tom  p.  and  all  are  the  same 
person.  The  system  only  allows  us  to  search  by  last  name,  first  name  or  company  name,  or  zip  code.  Middle  initial 
should  be  added. 

R4  comment:  CORALS  on  local  level  does  not  give  us  this  information,  we  verify  on  DNSP. 

R5  comment:  we  try  to  educate  our  customer  to  use  the  bin  number  assigned  to  them,  DRMS  should  try  to  come  up 
with  some  type  of  calling  card  sent  out  to  the  customers  once  they  register  to  have  their  name  and  bin  number  along 
with  their  valid  id  to  verify  up  front.  " 

This  is  the  first  I’ve  heard  of  any  changes  in  the  way  we  do  business  in  the  last  year;  no  idea  there  had  been  any 
changes.  If  we  do  not  know  the  new  bidder  process  why  would  I  be  able  to  answer  these  questions 


85 


I  do  not  understand  what  you  mean  by  NEW  bidder  registration.  We  have  only  had  one  way  that  I  know  of  since  the 
beginning  of  DAISY/DNSP.  What  is  this  new  bidder  registration  stuff?  I  know  only  one  way  to  register  bidders. 
You  could  really  shorten  bidder  registration  if  everything  you  needed  was  on  only  one  system  instead  of  two 
systems. 

What  new  bidder  registration  process? 

I  know  nothing  about  new  bidder  registration  process. 

Knew  the  above  info  was  in  the  works,  but  first  time  addressing  the  issues. 

R4  - 1  feel  that  DNSP  is  the  best  way  to  check  bidder  status 

"Received  e-mail  informing  of  a  test  in  which  the  way  we  input  names/addresses  will  change."  "Common  Sense"" 
will  follow  that  the  above  information  would  be  true.  We  at  the  DRMO  have  been  fussing  since  inception  of 
CORALS  because  there  was  no  interface  and  things  had  to  be  done  twice.  As  of  this  time,  I  have  heard  nothing 
concrete  regarding  test  results  and  if/when  interface  will  take  place.  The  e-mail  I  received  regarding  test  was  as  a 
result  of  being  included  as  a  CORALS  user." 

Comments  in  Question  6 

I  received  this  information  via  the  internal  data  on  the  DRMS  WEB  page.  His  questionnaire  does  not  allow  for  all 
the  potential  answers.  Specifically,  it  does  not  allow  for  information  to  be  gathered  via  personal  means,  working  on 
your  own. 

I  figured  these  two  points  of  information  myself  a  while  ago,  as  I  am  the  one  who  does  the  majority  of  checking  & 
registering  bidders.  Also,  I  am  the  only  one  to  add  bidders  to  our  local  list. 

EMAIL  from  DRMS  to  all  Chiefs.  In  turn  forward  to  all  Sales  Staff.  Been  with  DRMS  for  seven  years  and  from  the 

beginning  depended  on  DNSP  for  information . If  I  remember  right,  information  was  obtained  from  regulation  as  to 

how  to  check  bidders  and  what  info  to  use 

"I  was  already  full  aware  of  the  information  in  the  above  box,  before  the  questionnaire  was  sent  out." 

I  was  asked  for  my  input  to  the  bidder  registration  process  by  a  contact  at  DRMS-HQ  who  was  part  of  the  final 
product  sent  out  via  our  internal  Web  site.  I  feel  DRMS-HQ  has  a  great  untapped  resource  in  its  personnel  in  the 
field.  We  work  with  the  computer  programs  (DAISY,  DNSP,  BOSS,  etc)  daily.  We  know  the  'quirks’  in  the 
programs.  We  are  knowledgeable  about  the  regulations  and  our  jobs  as  well  as  the  needs  and  wants  of  our 
generators/customers.  We  may  even  have  internal  SOPs  which  fall  within  the  regulatory  requirements  yet  expedite 
the  final  product  which  saves  time  and  money  (time  is  money!)  and  provides  further  customer  satisfaction.  Customer 
satisfaction  includes  not  only  our  external  customers  (sales/generators/etc)  but  our  internal  customers  such  as  our  co¬ 
workers  and  HQ. 


Once  I  received  this  message  regarding  Bidder  Registration  via  e-mail  on  4/2, 1  contacted  the  Distribution  staff  and 
reviewed  the  process  and  information  with  them  on  4/6.  Since  we  don't  have  a  Distribution  Chief,  I  the  DRMO 
Chief  hold  meetings  with  various  branches  to  ensure  the  information  is  forwarded. 

I  don't  know  what  you  mean. 


I  am  not  sure  of  what  you  mean  by  new  registration.  There  is  only  one  way  I  know  of  to  register  and  that  is  found  in 
DRMS-1 4160.14  Vol  V.  Anything  else  I  have  not  heard  about.  I  learned  from  my  peers  how  sales  registrations  are 
to  be  processed  and  the  steps  to  follow  to  ensure  all  bidders  are  registered  and  not  indebted  nor  debarred.  Even 
before  notice  actually  came  down  as  to  how  we  can  simplify  registration  by  having  both  DNSP  and  DAISY  open  at 
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the  same  time  to  save  time,  we  have  been  doing  that  already.  I  feel  we  have  a  great  crew,  always  working  together  to 
find  new  and  better  ways  of  operating.  Thanks  / 

I  know  nothing  about  new  bidder  registration  process. 


Discussion 

While  there  may  have  been  some  contusion  as  to  whether  this  indeed  was  a  new  process  or  not,  many  respondents 
were  not  clear  about  what  the  desired  best  practice  is  for  bidder  registration.  The  document  issued  from 
Headquarters  contained  some  general  information  about  what  should  be  done.  However,  the  desired  behaviors 
required  some  advanced  skills,  such  as  complex  Boolean  searching  to  find  any  duplicate  names  in  the  databases  and 
the  simultaneous  use  of  two  different  applications  programs,  DNSP  and  CORALS.  The  results  of  the 
communications  baselining  survey  conducted  as  part  of  this  study  illustrated  that  field  staff  have  had  little  formal 
training  on  many  of  the  computer-based  tools  they  must  use  everyday.  It  is  recommended  that  future  process 
clarifications  that  require  the  mastery  of  advanced  computing  skills  be  accompanied  by  more  detailed  process 
descriptions,  including  relevant  examples  using  graphics  and  screen  captures  from  the  relevant  applications 
programs. 
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CAP  Survey  Form  #  2  -  AEDA  Policy  Guidance 


1 .  Before  receiving  this  questionnaire,  did  you  know  any  of  the  information  in  the  box  below? 
Please  check  each  of  the  items  you  knew. 


P  R1 

AEDA  inert  certifications  now  require  two  signatures  from  either  qualified  DoD  or  contractor  personnel. 

F  R2 

A  standard  memorandum  of  understanding  is  now  available  for  DRMO  use  with  all  of  their  AEDA 
generators. 

F  R3 

When  turned  in  to  the  DRMO,  containers  placed  on  or  banded  to  pallets  must  allow  for  visual  inspection  of 
all  containers. 

c  R4 

AEDA  Range  Residue  that  may  be  dangerous  to  public  health  and  safety  must  be  rendered  safe  and  the 
area  normally  containing  the  hazardous  material  must  be  opened  for  visual  inspection. 

P  R5 

Before  removal  of  AEDA  material,  qualified  individuals  will  brief  the  purchaser  and  DRMO  personnel  on 
the  responses/actions  necessary  in  the  event  that  live  or  suspected  live  AEDA  is  discovered. 

2.  If  you  had  the  information  in  the  box  but  you  heard  it  differently,  please  write  the  facts  you  heard  below. 
Use  the  reference  numbers  R1  -  R5  from  above. 


If  you  checked  above  that  you  knew  any  of  the  information ,  please  complete  the  questionnaire  by  providing  the 
information  requested  below. 

If  you  did  not  check  any  of  the  items  (you  did  not  know  any  of  the  information  in  the  box),  you  have  completed  the 


questionnaire .  Please  submit  the  questionnaire  using  this  button: 


Thank  you  very  much  for  your  cooperation . 


3.  From  whom  did  you  first  receive  the  information  in  the  box? 
Please  check  only  one. 


p. 

P 

P' 

C 


Superiors  above  your  immediate  superior? 
Immediate  superiors? 

Peers  (others  at  your  job  level)? 
Subordinates? 


4.  Where  were  you  when  you  first  received  the  information  in  the  box  above? 

Please  review  all  of  the  options  and  select  the  statement  that  best  represents  your  location.  Please  check 
only  one. 

P  I  was  working  within  my  designated  work  area  (at  my  desk,  board,  workstation,  etc.) 

P  I  was  working  within  my  department,  but  outside  of  my  designated  work  area. 

p  I  was  working  outside  of  my  department 

C  I  was  on  break  outside  of  my  department,  but  still  within  the  building  (lunch,  coffee  break,  etc.) 

C  I  was  out  of  the  building  and  not  working 
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5.  How  long  ago  did  you  first  receive  the  information  in  the  box? 
Check  the  approximate  time. 


Today 

1  -  2  days  ago 
3  -  4  days  ago 
5-6  days  ago 


1  -  2  weeks  a^o 


3  -  4  weeks  a^o 


5-6  weeks  ago 


6.  By  what  method  did  you  first  receive  the  information  in  the  box  above? 

Please  review  all  of  the  options  and  select  the  statement  that  best  represents  your  location.  Please  check 
only  one  of  the  following  methods. 

Written  or  visual  methods 

m 

•V  Personal  letter 

O  T 

Letter,  memo,  or  service  program 

icy' 

Electronic  mail  message 
P  Facsimile  (FAX) 

Newsletter 
^  Magazine 

Videotape  message 

lO 

Company  film 

O 

Public  newspaper  or  magazine 

w 

::  Records 


Our  internal  world  wide  web  site 
C 

An  external  world  wide  web  site 

Talking  or  sound  methods 

C 

Face-to-face  conversation 

i'C 

Talking  over  the  telephone 

C 

Talking  (and  listening)  in  a  small  group  of  two  or  more 

r  ■ 

Attending  an  organized  meeting,  briefing  or  conference 
■O 

Overhearing  what  someone  else  said 

C  ' 

Radio  or  television 


Miscellaneous 

r  ' 


I  am  the  subject/source  of  the  message 
Other,  (Please  explain.) 
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Thank  you  very  much  for  your  cooperation. 
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DRMS  Communication  Effectiveness  Survey  #2 
Ammunition,  Explosives,  and  Other  Dangerous 
Articles  (AEDA)  Policy  Guidance 

ERIM 

May  17, 1999 


In  early  April,  the  publication  of  revised  guidance  for  the  processing  of  AEDA  and  Range  Residue  communicated 
from  DRMS  HQ  to  the  DRMS  Vice  Presidents,  Zone  Managers,  and  DRMO  Chiefs.  This  policy  change  should 
have  been  communicated  down  the  following  hierarchy: 

The  DRMO  Chief  to: 

•  The  Property  Management  Branch  to  the  Demil  Coordinator; 

•  The  Distribution  Branch  Chief  to  the  Marketing  Specialist(s);  and 

•  The  Environmental  Branch  Chief 

The  original  email  message  requested  that  all  affected  parties  review  the  details  of  the  change  available  on  both  the 
DEMIL  page  of  the  DRMS  Internal  World  Wide  Web  site  and  on  the  DoD  Demilitarization  and  Trade  Security 
Controls  Web  site. 

We  solicited  survey  respondents  emails  sent  to  the  Zone  Managers  and  DRMO  Chiefs.  The  Zone  Managers  were 
provided  a  URL  to  a  password-protected  WWW  site.  Their  user  name,  password,  and  code  number  identified  them 
by  Zone.  Chief  received  a  similar  email,  requesting  that  they  respond  and  to  forward  a  similar  request  to  the 
appropriate  staff  members.  Indirect  contact  was  necessary  since  HQ  could  not  provide  direct  email  contact  with  the 
specific  staff  categories.  For  this  same  reason,  we  can  only  estimate  the  possible  number  of  respondents.  Based  on 
12  Zone  Managers  and  83  DRMOs,  we  estimate  that  there  could  be  approximately  510  respondents. 

Results 

These  results  are  based  on  91  respondents.  Overall,  this  is  an  18%  response  rate.  This  is  reasonable  for  a  survey  of 
this  type,  but  is  much  lower  than  for  our  first  survey.  There  are  several  possible  explanations  for  this  difference. 

The  first  survey  was  sent  out  from  the  DRMS  Commanders  email  address,  the  second  from  a  HQ  staffer.  The  second 
solicitation  was  sent  out  soon  after  the  first  and  may  not  have  been  distinguished  as  a  separate  survey.  Finally,  some 
respondents  may  have  simply  declined  to  be  surveyed  again. 

Responses  rate  by  zone  vary  widely.  We  arbitrarily  numbered  the  12  Zones  from  1  to  12.  The  rates  for  each  Zone 
are  shown  below. 


Zo  ne 


1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

Total  Rate 

58 

32 

16 

17 

16 

11 

16 

23 

42 

28 

6 

20 

Chiefs 

100 

50 

17 

43 

33 

33 

33 

33 

67 

44 

17 

36 

Zone  Mgrs  42 

Originally  we  wanted  to  track  the  response  down  the  chain  of  command,  to  see  how  well  each  message  traveled  from 
the  Zone  Manager  to  their  Chiefs,  then  from  each  Chief  to  their  first  line  supervisors,  and  finally  from  those 
supervisors  to  their  subordinates.  As  stated  above,  the  best  we  could  do  with  the  email  contact  information  available 
was  to  get  to  each  Chief  specifically  and  then  ask  them  to  pass  a  separate  message  to  their  first  lines  and  the 
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appropriate  subordinates.  We  assigned  code  numbers  to  each  respondent  that  would  identify  their  position  in  the 
chain.  Unfortunately,  our  instructions  must  not  have  been  explicit  enough  because  many  Chiefs  passed  on  their 
message  (with  their  code  number)  to  their  subordinates.  Thus,  our  database  contains  data  apparently  from  multiple 
Chiefs  from  each  DRMO.  (From  their  responses  and  other  information  captured  with  the  survey  data,  we  know  these 
multiple  responses  probably  did  not  come  from  the  same  person.)  In  our  last  survey,  we  tried  a  different  approach  to 
avoid  this  problem. 

The  first  question  includes  five  factual  statements  about  the  Bidder  registration  process.  Ideally,  the  respondents 
should  know  all  five.  Scoring  "five  out  of  five"  would  yield  a  score  of  100%,  "four  out  of  five"  80%  and  so  on.  As 
can  be  seen  in  the  graph  below,  a  majority  of  respondents  did  know  about  the  changes  in  the  AEDA  policy. 


Respondent  Scores 


This  message  was  intended  to  be  sent  from  HQ  simultaneously  to  the  Zone  Managers  and  DRMO  Chiefs,  and  then  to 
the  Property  Management  Branch  Chief  (to  Demil  Coordinators),  the  Distribution  Branch  Chief  (to  the  Marketing 
Specialists),  and  the  Environmental  Branch  Chief.  Given  this,  you  would  expect  most  respondents  to  hear  the 
message  from  superiors.  This  is  reflected  in  the  chart  below.  Many  Chiefs  probably  passed  the  HQ  message  directly 
to  their  Chiefs  and/or  subordinates. 
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A  majority  of  respondents  received  the  information  in  their  immediate  work  area. 


Where  Received 


B  In  work  area 
■  In  department 

□  Out  of  department 

□  Not  working 


1  he  original  message  from  HQ  about  the  AEDA  policy  revision  was  sent  out  in  the  first  week  of  April.  This  survey 
was  conducted  during  the  first  two  weeks  of  May.  As  a  manager,  you  would  hope  that  most  people  received  the 
message  soon  after  you  sent  it,  i.e.,  ”3-4  weeks  ago"  or  “5-6  weeks  ago”.  The  graph  below  shows  that  82%  (64% 
plus  18%)  did. 
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When  Received 


B Today 

■  1-2  weeks  ago 
01-2  days  ago 
:Q3-4  weeks  ago 

■  3-4  days  ago 
05-6  weeks  ago 


This  process  change  was  communicated  to  the  Zone  Managers  and  DRMO  chiefs  using  email  In  the  message  Aey 
were  told  to  go  to  one  of  two  W  WW  sites  (one  internal  and  one  externa!)  to  get  more  details.  The  DRMO  Chiefs 
could  then  provide  the  information  to  their  staff  using  their  normal  channels.  As  you  can  see  in  die  chart  below  mos 
DRMS  staff  received  the  message  via  email 


Method 


8%  11% 


32% 


15% 


3%  5% 


H  Letter,  memo 

■  Email 

□  FAX 

□  Internal  WWW 

■  Face-to-face 
g  Telephone 

■  Small  Group 

□  Organized  meeting 

■  Other 


One  possible  down  side  of  using  a  wide  variety  of  mechanisms  is  that  there  are  more  opportunities  for  die  message  to 
be  misinterpreted. 


In  this  survcv.  there  are  mo  opportunities  for  respondents  to  provide  input  in  their  own  words  question  2  (  did  you 
hear  the  information  in  question  differently?!  and  question  6  (“By  what  other  means  did  you  first  receive  the 
information  in  question?”).  For  completeness,  we  include  those  comments  modified  only  for  spelling  and  to  remove 
comments  that  could  identify  specific  respondents. 
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Comments  in  Question  2 

I  have  reviewed  the  above  question,  have  seen  some  e-mail  regarding  some  of  the  question,  the  question  could  have 
been  written  a  little  better  ...  and  took  a  lot  of  time  to  research  to  make  sure  I  didn't  miss  something  ...  the  second 
group  of  question  were  bad! ! ! !  (They  were  poorly  written)  if  you  want  to  talk  about  it  call  «DRMO  Chief  deleted 
-  we  talked  to  him  at  length  about  our  work» 

This  has  been  standard  practice  at  our  DRMO  for  approximately  3  years. 

Much  of  the  above  info  has  been  received  over  time.  Received  mostly  from  Host  AEDA  officials  and  in  a  slightly 
different  wording  but  the  same  basic  information. 

I  do  understand  the  requirement  to  have  two  signatures  on  the  inert  certification  but  do  not  understand  why. 

I  was  under  the  impression  that  if  material  is  turned  in  with  signatures  no  visual  inspection  was  necessary.  Further,  it 
is  my  understanding  that  we  do  not  open  containers  or  un-band  material,  as  we  have  no  one  qualified  to  examine  the 
material  for  live  rounds  anyway. 

R1  - 1  thought  only  one  signature  was  required. 

R5  -  The  initial  responding  DoD  Component  will  serve  as  the  DoD  representative  to  ensure  that  all  necessary  actions 
are  taken  to  remedy  the  situation  and  to  ensure  that  all  live  or  suspected  live  ordnance  is  either  destroyed  or  returned 
to  DoD  control. 

R3  -  Last  letter  received,  can  request  generators  configure  empty  containers  this  way,  but  is  not  mandatory  until 
change  approved. 

Comments  in  Question  6 

Actually,  the  information  I  received  on  above  checked  box  was  during  training  at  DRMS. 

"This  is  a  fuzzy  memory.  I  do  not  really  recall  'when'.  This  is  a  case  of  "I  think  I  recall  seeing/hearing  something  on 
that".  I  don't  think  anyone  can  keep  current  on  the  very  wide  variety  of  guidance  we  get,  and  I  tend  to  seek  out 
information  when  faced  with  a  situation.  If  faced  today  with  a  situation  on  AEDA,  I'd  be  saying  to  myself:  "I  think  I 
saw/heard  something  recently  on  that",  and  I'd  begin  looking  for  hard  data  before  doing  anything." 

Email  from  DRMS-TSD,  which  led  me  to  the  DoD  Demilitarization  and  Trade  Security  Control  Web  Site.  Info  was 
downloaded  and  also  disseminated  to  applicable  functional  areas  at  the  DRMO. 

I  did  not  get  all  of  the  information  in  the  box  in  the  same  manner.  Most  of  it  was  via  email,  but  some  was  via 
telephone  and  official  memorandum.  Also  most  of  the  information  I  have  known  for  much  longer  than  5-6  weeks, 
some  for  over  a  year. 

The  chief  called  a  meeting  when  he  found  or  got  word  that  this  info  was  out. 

Do  not  recall. 

As  mentioned  above,  most  of  the  info  has  been  received  over  a  long  period  of  time. 

Verbal  conversation  with  Demil  Coordinator  at  «DRMO  name  deleted»  who  recommended  that  I  check  the  Web 
for  the  latest  DRMS  Letter  regarding  AEDA  policy.  This  I  attempted  to  do  but  had  to  go  back  to  him  for  further 
clarification  of  where  to  find  Letter. 
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"What  is  DRMS  HQ  going  to  do  when  there's  no  one  left  in  the  field  to  take  the  blame?  But  then  again,  WHO 
CARES!!!” 

Source  was  taken  from  regulations  (4160.21M) 

Discussion 

This  change  was  disseminated  much  faster  than  in  our  previous  survey,  with  a  higher  comprehension  rate.  This  is 
despite  the  fact  that  these  changes  were  buried  in  revised  DoD  standards.  The  DRMO  staff  should  be  commended 
on  their  ability  to  decipher  the  changes  (something  this  researcher  could  not  do  as  easily).  If  the  specific  changes 
were  highlighted  as  part  of  the  communication,  it  might  be  easier  on  all  of  those  who  have  to  leam  about  them.  (I 
tried  to  do  a  side-by-side  comparison  of  the  old  and  new  versions  but  the  cited  Web  pages  did  not  provide  the 
necessary  information.)  It  is  also  worth  noting  that  many  people  knew  that  these  changes  were  in  the  works  for  some 
time. 

One  point  made  by  a  respondent  is  worth  noting,  however.  In  the  original  message,  the  respondents  were  told  to  go 
to  the  DRMS  DEMIL  page  to  find  the  necessary  information.  It  took  four  screens  of  information  to  get  to  that  point. 
One  ready  benefit  of  the  WWW  is  that  it  is  possible  to  directly  address  any  resource  (i.e.,  URL)  on  the  Web.  Future 
communications  could  make  it  easier  on  receivers  if  the  direct  URLs  were  provided. 
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CAP  Survey  Form  #3  -  Electrical  and  Electronic  Scrap 


1 .  Before  receiving  this  questionnaire,  did  you  know  any  of  the  information  in  the  box  below? 
Please  check  each  of  the  items  you  knew . 


t"  R1 

The  main  objective  of  the  electrical  and  electronic  scrap  demanufacturing  contract  awarded  December  30, 
1998  is  higher  proceeds. 

n  m 

Use  of  the  demanufacturing  contracts  is  mandatory  in  the  continental  U.S. 

C  R3 

Sale  of  electronic  scrap  by  local  or  zone  methods  is  still  authorized. 

n  R4 

Scrap  property  destined  for  the  demanufacturing  must  be  handled  like  usable  property. 

o  R5 

Usable  property  in  critical  Federal  Supply  Classes  (FSCs)  to  be  downgraded  to  scrap  and  removed  by  the 
demanufacturing  contractors  is  exempt  from  the  mutilation  requirements  for  critical  FSCs. 

2.  If  you  had  the  information  in  the  box  but  you  heard  it  differently,  please  write  the  facts  you  heard  below. 
Use  the  reference  numbers  R1  -  R5  from  above. 


If  you  checked  above  that  you  knew  any  of  the  information,  please  complete  the  questionnaire  by  providing  the 
information  requested  below. 

If  you  did  not  check  any  of  the  items  (you  did  not  know  any  of  the  information  in  the  box),  you  have  completed  the 
questionnaire.  Please  submit  the  questionnaire  using  this  button: 


Thank  you  very  much  for  your  cooperation. 


3.  From  whom  did  you  first  receive  the  information  in  the  box? 
Please  check  only  one . 


C 

C 

c 

c 

T, 


Superiors  above  your  immediate  superior? 
Immediate  superiors? 

Peers  (others  at  your  job  level)? 
Subordinates? 

None  of  the  above? 


4.  Where  were  you  when  you  first  received  the  information  in  the  box  above? 

Please  review  all  of  the  options  and  select  the  statement  that  best  represents  your  location.  Please  check 
only  one. 

O  I  was  working  within  my  designated  work  area  (at  my  desk,  board,  workstation,  etc.) 

O  I  was  working  within  my  department,  but  outside  of  my  designated  work  area. 

JO  I  was  working  outside  of  my  department 

!.C  I  was  on  break  outside  of  my  department,  but  still  within  the  building  (lunch,  coffee  break,  etc.) 
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I  was  out  of  the  building  and  not  working 


5 .  How  long  ago  did  you  first  receive  the  information  in  the  box? 
Check  the  approximate  time. 


P  Today 

W: 

1  -  2  weeks  ago 

^  1-2  days  ago 

3  -  4  weeks  ago 

3-4  days  ago 

m 

5  -  6  weeks  ago 

P  5-6  days  ago 

6.  By  what  method  did  you  first  receive  the  information  in  the  box  above? 

Please  review  all  of  the  options  and  select  the  statement  that  best  represents  your 
only  one  of  the  following  methods. 

Written  or  visual  methods 

fe- 


•o:; 

p 

D 

C 

■O; 

p' 

C 

h 

p 

jb 

b 


Personal  letter 

Letter,  memo,  or  service  program 
Electronic  mail  message 
Facsimile  (FAX) 

Newsletter 
Magazine 
Videotape  message 
Company  film 

Public  newspaper  or  magazine 
Records 

Our  internal  world  wide  web  site 
An  external  world  wide  web  site 


Talking  or  sound  methods 

c 


o. 

:o: 

c 

o 


Face-to-face  conversation 
Talking  over  the  telephone 

Talking  (and  listening)  in  a  small  group  of  two  or  more 
Attending  an  organized  meeting,  briefing  or  conference 
Overhearing  what  someone  else  said 
Radio  or  television 


Miscellaneous 

C 


I  am  the  subject/source  of  the  message 


Other,  (Please  explain.) 


location.  Please  check 
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7.  Please  use  this  space  to  provide  any  comments  on  this  survey,  the  management  communication  in  question, 


etc. 


Thank  you  very  much  for  your  cooperation. 
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DRMS  Communication  Effectiveness  Survey  #3 
Electrical  and  Electronic  Scrap  Demanufacturing 
Contract  Policy  Letter 

ERIM 

May  17, 1999 


In  late  April,  DRMS  HQ  published  a  memorandum  to  the  National  Region  Manager,  National  Zone  Managers,  and 
the  DRMO  Chiefs  about  the  demanufacturing  contracts  for  electrical  and  electronic  scrap.  This  policy  change 
should  have  been  communicated  down  the  following  hierarchy: 

The  DRMO  Chief  to: 

•  The  Property  Management  Branch  to  the  Demil  Coordinator,  Receiving  Coordinator(s)/Schedulers,  and 
Receivers. 

•  The  Distribution  Branch  Chief  to  the  Marketing  Specialist(s);  and 

•  The  Environmental  Branch  Chief 

We  solicited  survey  respondents  emails  sent  to  the  Zone  Managers  and  DRMO  Chiefs.  The  Zone  Managers  were 
provided  a  URL  to  a  password-protected  WWW  site.  Their  user  name,  password,  and  code  number  identified  them 
by  Zone.  Chief  received  a  similar  email,  requesting  that  they  respond  and  to  forward  a  similar  request  to  the 
appropriate  staff  members.  Indirect  contact  was  necessary  since  HQ  could  not  provide  direct  email  contact  with  the 
specific  staff  categories.  For  this  same  reason,  we  can  only  estimate  the  possible  number  of  respondents.  Based  on 
12  Zone  Managers  and  83  DRMOs,  we  estimate  that  there  could  be  approximately  510  respondents. 


Results 

These  results  are  based  on  92  respondents.  Overall,  this  is  an  18%  response  rate.  This  is  reasonable  for  a  survey  of 
this  type,  but  is  much  lower  than  for  our  first  survey.  There  are  several  possible  explanations  for  this  difference. 

The  first  survey  was  sent  out  from  the  DRMS  Commanders  email  address,  the  second  from  a  HQ  staffer.  The  second 
solicitation  was  sent  out  soon  after  the  first  and  may  not  have  been  distinguished  as  a  separate  survey.  Finally,  some 
respondents  may  have  simply  declined  to  be  surveyed  again. 

Responses  rate  by  zone  vary  widely.  We  arbitrarily  numbered  the  12  Zones  from  1  to  12.  The  rates  for  each  Zone 
are  shown  below. 


Zone 


1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

Total  Rate 

31 

32 

3 

22 

6 

13 

23 

26 

35 

35 

26 

13 

Chiefs 

60 

67 

17 

.  43 

17 

33 

17 

50 

50 

56 

17 

36 

f 

Zone  Mgrs  33 


Originally  we  wanted  to  track  the  response  down  the  chain  of  command,  to  see  how  well  each  message  traveled  from 
the  Zone  Manager  to  their  Chiefs,  then  from  each  Chief  to  their  first  line  supervisors,  and  finally  from  those 
supervisors  to  their  subordinates.  As  stated  above,  the  best  we  could  do  with  the  email  contact  information  available 
was  to  get  to  each  Chief  specifically  and  then  ask  them  to  pass  a  separate  message  to  their  first  lines  and  the 
appropriate  subordinates.  We  assigned  code  numbers  to  each  respondent  that  would  identify  their  position  in  the 
chain.  Unfortunately,  our  instructions  must  not  have  been  explicit  enough  because  many  Chiefs  passed  on  their 
message  (with  their  code  number)  to  their  subordinates.  Thus,  our  database  contains  data  apparently  from  multiple 
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Chiefs  from  each  DRMO.  (From  their  responses  and  other  information  captured  with  the  survey  data,  we  know  these 
multiple  responses  probably  did  not  come  from  the  same  person,) 

The  first  question  includes  five  statements  about  the  demilitarization  contracts.  Three  were  “true'*  and  two  were 
intentionally  made  “false”,  Ideally,  the  respondents  should  know  all  five.  Scoring  "five  out  of  five"  would  yield  a 
score  of  100%,  "four  out  of  five"  80%  and  so  on.  As  can  be  seen  in  the  graph  below,  the  scores  are  good.  Most  of 
the  respondents  knew  something  about  the  new  policy,  with  57%  of  respondents  “passing”  the  “test”.  However,  the 
fact  that  1 8  respondents  (almost  20%)  had  either  yet  to  hear  about  the  policy  or  knew  none  of  the  “facts”  should  be  a 
cause  for  some  concern. 


This  message  was  intended  to  be  sent  from  HQ  simultaneously  to  the  Zone  Managers  and  DRMO  Chiefs,  and  then  to 
the  Property  Management  Branch  Chief  (to  Demil  Coordinators,  Receiving  Coordinator(s)/Scheduiers,  and 
Receivers),  the  Distribution  Branch  Chief  (to  the  Marketing  Specialists),  and  the  Environmental  Branch  Chief. 

Given  this,  you  would  expect  most  respondents  to  hear  the  message  from  superiors.  This  is  reflected  in  the  chart 
below.  Many  Chiefs  probably  passed  the  HQ  message  directly  to  their  Chiefs  and/or  subordinates. 
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A  majority  of  respondents  received  the  information  in  their  immediate  work  area. 


Where  Received 


|  □  In  work  area 
■  tn  department 
□  Out  of  department  j 
•  O  On  break 


The  original  message  from  HQ  about  the  demilitarization  contracts  was  sent  out  on  29  April.  This  survey  was 
conducted  during  the  first  two  weeks  of  May.  As  a  manager,  you  would  hope  that  most  people  received  the  message 
soon  after  you  sent  it,  i.e.,  "1-2  weeks  ago”  or  “5-6  weeks  ago”.  It  is  not  clear  why  a  vast  majority-  of  the  respondents 
(67%)  said  they  received  this  information  5-6  weeks  ago,  unless  there  was  a  previous  communication  about  this 
topic.  (The  responses  to  the  open-ended  question  provide  the  answer;  many  people  in  the  field  have  been  hearing 
about  this  for  months.) 


When  Received 


3%  9% 


16% 


67% 


;  ©Today 
|  ■  1-2  weeks  ago 
[a  1-2  days  ago 
)q  3-4  weeks  ago 
■  5-6  weeks  ago 
|  S  5-6  days  ago 


102 


This  process  change  was  communicated  to  the  Zone  Managers  and  DRMO  chiefs  using  FAX.  The  DRMO  Chiefs 
could  then  provide  the  information  to  their  staff  using  their  normal  channels.  As  you  can  see  in  the  chart  below  most 
DRMS  staff  received  the  message  via  email. 


One  possible  down  side  of  using  a  wide  variety  of  mechanisms  is  that  there  are  more  opportunities  for  the  message  to 
be  misinterpreted. 

In  this  survey,  there  are  three  opportunities  for  respondents  to  provide  input  in  their  own  words: 

•  Question  2  (“did  you  hear  the  information  in  question  differently?”) 

•  Question  6  (“By  what  other  means  did  you  first  receive  the  information  in  question?”). 

•  Question  7,  an  open  ended  item  added  for  this  last  survey  to  give  respondents  a  place  to  comment  on  the  survey, 
or  just  about  anything  else. 

For  completeness,  we  include  those  comments  modified  only  for  spelling  and  to  remove  comments  that  could 
identify  specific  respondents. 

Comments  in  Question  2 

In  this  question,  respondents  were  allowed  to  comment  on  the  “facts”  listed  in  the  first  question.  For  your  reference, 
these  are  the  “facts”  used  as  part  of  this  survey: 

Rl.  The  main  objective  of  the  electrical  and  electronic  scrap  demanufacturing  contract  awarded  December  30,  1998 
is  higher  proceeds.  (This  is  a  FALSE  statement.) 

R2.  Use  of  the  demanufacturing  contracts  is  mandatory  in  the  continental  U.S. 

R3.  Sale  of  electronic  scrap  by  local  or  zone  methods  is  still  authorized.  (This  is  a  FALSE  statement.) 

R4.  Scrap  property  destined  for  the  demanufacturing  must  be  handled  like  usable  property. 

R5.  Usable  property  in  critical  Federal  Supply  Classes  (FSCs)  to  be  downgraded  to  scrap  and  removed  by  the 
demanufacturing  contractors  is  exempt  from  the  mutilation  requirements  for  critical  FSCs. 

R3  sale  NOT  authorized,  per  DRMS-LM  memo  dated  4/29/99 

R3  sale  of  electronic  scrap  is  NOT  A  UTHORIZED,  PER  DRMS  MEMO  DA  TED  29  APR  99" 
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R4/R5.  I  didn 't  hear  anything  differently,  I just  wasn't  aware  of  those  facts,  but  my  employees  do. 

Rl.  This  Contract  is  a  service  contract  and  it  cost  DRMS  money.  We  do  not  get  proceeds 

R3.  Sale  of  electronics  (le.t  computers  turning  from  a  federal  agency  must  be  sold  on  a  local  per  DRMS. 

Rl  -  Thought  environmental  considerations  were  the  main  objective. 

R4  - 1  was  under  the  impression  that  scrap  was  scrap  and  Demil  was  treated  as  usable  property  for  accountability. 
R2-  as  long  as  you  don't  have  a  term  D4R  contract  in  place. 

R3  Local  or  zone  sale  of  electronic  scrap  is  not  authorized. 

The  main  objective  for  electronic  scrap,  if  this  includes  AE,  should  be  RTD  to  UNICOR.  Why  should  we  give  it  to 
someone,  like  the  contractor,  when  we  can  save  Tax  dollars  through  the  UNICOR  program.  The  DRMO  receives 
higher  proceeds  through  RTD  than  the  contractor. 

Rl:  The  main  objective  of  demanufacturing  contract  is  to  reduce  the  volume  of  scrap  thus  resource  reduction. 

Rl:  I  thought  the  demanufacturing  contract  was  a  service  contract,  meaning  we  pay  for  the  service,  not  a  proceeds 
contract,  where  they  pay  us. 

R3,4,5: 1  have  no  idea" 

R3-I  thought  ALL  electronic  scrap  will  go  to  the  Demanufacturing  Contract. 

R4  -  Why  would  scrap  be  handled  like  usable?" 

Rl  -I  thought  the  main  objective  of  the  demanufacturing  contract  was  environmental  concerns.  If  it  is  higher 
proceeds,  they  are  way  of  track.  I  can  get  higher  proceeds  through  local  or  zone  sale. 

R3  .  /  just  received  a  letter  yesterday  from  DRMS-LM ’  dated  April  29,  1999  saying  local  or  zone  sale  is  not 
authorized. 

R4  -  This  is  brand  new  guidance  (April  29  letter)  that  contradicts  previous  guidance  and  is  also  unworkable. " 

I  heard  that  the  main  objective  was  more  environmental  in  nature.  And  a  cost  avoidance  for  third  party  sites.  Also , 
my  sales  staff  has  been  told  that  use  of  this  contract  vice  sales  is  mandatory. 

Rl  -  The  main  objective  is  pollution  prevention. 

R3  -  Local  or  Zone  sales  are  NOT  authorized. " 

Sale  of  electronic  scrap  by  local  or  zone  sales  was  not  authorized. 

Rl  -  The  main  objective  was  to  relieve  the  DRMO  of  doing  demil  and  mutilation,  higher  proceeds  seemed  to  be 
secondary. 

R3  -  Sale  of  electronic  scrap  by  local  or  zone  method  is  NOT  authorized  any  longer. " 

Rl.  I  heard  it  was  a  primarily  for  the  environmental  concern  regarding  D4R,  capacitors,  etc.  The  contractor  would 
be  responsible  for  this. 

R3.  We  are  not  allowed  to  sell  electronic  scrap  on  national  or  local  as  of  29  Apr  99. " 
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Rl  -  The  main  objective  is  to  recycle  the  material  content  of  unusable  or  obsolete  DoD-owned  electrical  and 
electronic  equipment  in  an  environmentally  responsible  manner ,  rendering  any  non-commercial  items  completely 
unusable  by  the  process  of  demilitarizing  the  property . 

R3  -  Only  if  a  contract  was  in  place . 

R4  -  This  statement  is  false. 

I  believe  there  is  some  confusion  about  what  the  contractors  are  looking  for — one  appears  to  want  scrap  and  the 
other  usable.  Possibly,  it  is  clear  to  others  but  not  to  myself  as  yet. 

Rl  Main  objective  is  to  reduce  DRMO  workload/man-years. 

R2  Not  mandatory  for  many  DRMOs . 

R3  Depends  on  circumstances,  generally  yes. 

R4  In  a  talk  with  Demanufacturing  Contractor,  he  says  yes,  but  practical  side,  no.  Property  is  scrap. 

Have  just  been  informed  by  my  Sales  Writer  that  attended  training  in  Battle  Creek  that  the  sale  of  electronic  scrap 
on  local  sale  is  now  prohibited;  however,  to  my  knowledge,  I  have  not  seen  that  instruction  in  writing. 

Rl  demanufacturing  contract  is  for  environmental  reasons  not  proceeds. 

Rl=  Proceeds  were  not  the  primary  reason,  environmental  concems/Liability  was  the  primary  reason 

R3=  Sale  of  electronic  scrap  by  local  or  zone  methods  was  not  authorized  per  DRMS-LM  leter  dated  4/29/99  same 
subj. 

R3-Know  that  the  sale  of  electronic  scrap  on  existing  contracts  was  still  okay,  but  no  new  referrals. 

R4-Did  not  know  that  we  were  to  handle  the  scrap  as  usable,  the  referral  said  that  we  could  order  roll-off 
containers  which  led  us  to  believe  that  we  could  dump  from  hoppers. 

R4  Just  last  week  I  learned  through  my  zone  manager  that  this  scrap  could  not  go  out  in  hoppers  but  must  be 
packaged  similar  to  usable  property.  A  few  days  ago  I  got  something  in  writing  verifying  that. 

R3  electronic  scrap  can  not  be  sold  on  local  or  zone  sales. 

R3.  Instruction  sent  out  in  February  99  states  under  scope:  " Electronic  material  that  has  failed  sales  efforts  and  is 
eligible  to  be  downgraded  to  scrap,  except  for  Precious  Metals  bearing  material  earmarked  for  the  precious  metal 
program  ".  Also  states  "Electronic  material/scrap  (i.e.  D4R)  that  is  not  currently  on  an  existing  sales  contract If 
this  is  what  is  supposed  to  be  offered  on  the  DMFG  contract  how  could  you  still  sell  it  by  local  or  zone  methods? 

R4..The  same  instruction  stated  above  indicates:  "If  sufficient  weight  is  available,  prepare  material  for  shipment. 
Follow  normal  shipping  procedures  (e.g.,  banding  to  pallets,  shrink  wrapping,  placing  in  tri-walls,  etc.,  hopper 
dump  or  crane  load  open  top  conveyance. "  Totally  contradicts  R4. 


Comments  in  Question  6 

These  are  the  list  of  other  “sources”  of  information. 
Environmental  Branch  Chief 
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Various  sources 


Even  though  J  had  received  email  traffic  concerning  the  Demanufacturing  contract,  I  did  not  fully  understand  the 
procedures  until  I  had  face-to-face  conversation  with  XXX  at  DRMS. 

"My  supervisor  informed  all  employees  in  our  weekly  meeting  about  the  dmfg  contract. " 

This  subject  came  up  during  an  informal  meeting  to  discuss  other  subjects  in  which  1  am  directly  involved. 

Comments  in  Question  7 

These  comments  could  be  about  the  communication  in  question,  the  survey  itself,  or  any  other  topic. 

I  think  this  could  be  explained  in  better  way.  The  questions  are  too  wide  in  range 

I  knew  about  this  project  very  early  because  DMC  is  right  up  the  road  from  both  the  DRMOs  I  command.  XXX  and 
YYY.  Iam  also  a  COTR  on  both  contracts. 

This  is  a  waste  of  time,  what  difference  does  it  make  where  you  where  when  the  lights  went  out  in  Georgia? 

DRMS  does  a  lousy  job  on  keeping  their  employees  briefed  on  all  the  initiatives  that  are  ongoing.  More 
efforts  need  to  be  brought  forward  toward  this  end. 

I  question  that  the  information  is  getting  to  the  actual  personnel  doing  to  work,  i.e.,  scrap  yard  personnel  or  those 
that  may  downgrade  items. 

Some  of  the  information  in  the  box  was  sent  out  several  months  ago.  Other  information  is  brand  new  and 
contradicts  previous  guidance.  R3  in  the  box  is  false  according  to  most  recent  guidance.  Also,  the  source  of  the 
information  varied.  Most  was  by  formal  memo,  some  was  by  email.  Most  of  it  was  after  the  fact.  There  was  almost 
no  information  provided  to,  or  solicited  from,  the  field  prior  to  the  contract  being  awarded.  Changes  are  still  being 
made  without  any  field  input. 

Question  #3  -  received  the  information  from  various  sources.  Peers  on  a  conference  call  and  the  same  and 
additional  information  in  a  letter  signed  by  a  staff  EVP.  Question  # 4  has  not  been  applicable  in  any  of  he  surveys. 

I  feel  this  is  going  to  cost  the  Government  way  too  much  money. 

This  info  should  have  been  provided  months  ago,  when  we  first  started  accumulating  this  scrap  for  the  contract.  We 
did  not  know  it  had  to  be  treated  and  stored  as  usable. 

Possibly  you  should  accept  more  than  one  answer  for  a  question-only  one  answer  is  limiting,  but  maybe  that  is 
your  intent.  Also  possibly  allow  more  opportunities  to  briefly  offer  explanations/comments. 

I  have  also  been  told  that  just  because  a  PCB  suspected  electronic  device  may  be  put  into  the  Demanf.  items.  Is  this 
so?  Like  boxes  of  already  removed  PCB  suspect  transformers  and  capacitors  from  electronic  cabinets,  etc. 

This  Contract  has  to  be  the  worst  executed  since  1  have  been  in  DRMS  (15  years).  No  one  in  the  field  was  involved 
in  the  process  until  after  it  was  awarded.  After  it  was  awarded,  the  contract  was  taken  off  the  WWW  I  feel  in  an 
attempt  to  keep  the  field  from  reading  the  contract.  To  date  the  work  instruction  is  very  unclear  and  everything  I 
have  learned  is  from  fellow  DRMO  workers  and  not  from  the  appropriate  official  personnel.  If  someone  can  explain 
to  me  how  we  will  make  more  money  selling  scrap  to  deman  (considering  the  preparation  we  are  supposed  to  be 
doing)  when  were  making  money  selling  it  at  the  DRMOs.  I  feel  we  are  being  forced  to  lie  on  the  inventory  in  that 
we  are  referring  demil  required  property  as  a  usable  line  item  when  indeed  it  has  been  scrapped  and  DRMS  is 
selling  a  usable  item  as  scrap.  Again,  it  is  the  worst  executed  contract  to  date  (outdoing  the  auctioning). 
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The  Demanufacturing  contract  was  very  confusing  to  the  personnel  in  the  field.  We  were  not  made  aware  of  the 
specifics  other  than  it  was  going  to  happen.  I  thought  the  contractor  was  going  to  pay  us,  not  the  other  way. 

It  turns  out  that  we  are  paying  to  remove  (i.e.  service  contract  type)  and  then  the  ' government '  will  be  reimbursed 
for  something  later  down  the  line.  How  is  that  going  to  work?  We  need  to  get  a  better  feel  of  what  is  happening. 
DRMS  should  not  put  together  something  without  letting  someone  in  the  field  know  what  is  going  on  and  then  tell 
DRMO  personnel  that  they  have  to  do  it  without  any  kinds  of  explanations  and/or  initial  guidance.  When  we  were 
told  this  was  a  scrap  contract,  we  thought  we  could  handle  as  scrap.  All  other  D4R  contracts  that  were  handled  by 
sales  contract  were  treated  as  scrap,  loaded  with  front  end  loaders  or  dumped  from  hoppers.  Even  though  this  is 
not  called  D4R  anymore,  we  all  know  that  is  exactly  what  it  is. 

There  have  been  many  updates  to  this  contract.  We  just  went  online  with  RCP  in  April,  we  delivered  our  first  group 
of  CV  property  in  mid  March,  we  have  two  facilities  (DRMOs).  We  just  closed  but  are  working  as  forward  receiving 
activities,  we  have  the  demil  center  and  are  trying  to  get  all  the  off-site  DRMOs  on  line  to  start  turning  in  to  us. 

Plus  we  are  getting  ready  to  downsize  by  more  than  1/3  of  our  workforce  (from  29  to  17)  by  1  October,  and  I've  got 
more  e-mail  than  I  can  handle.  It's  no  wonder  to  me  that  we  have  a  hard  time  keeping  up  with  what  is  going  on. 

,fthe  instructions  for  processing,  storage,  handling,  and  shipments  need  to  be  fine  tuned  so  everybody  is  working 
together  correctly  .  In  house  SOP.  There  are  just  to  many  options  in  this  contract.  I  will  be  working  this  material 
so  this  comment  is  critical  to  me" 

Where  are  you  going  with  this/what  is  the  purpose?  If  this  process  was  mandated,  there  are  sure  some  puppets 
out  there.  Also,  someone  had  better  have  another  talk  with  Bucks  Bunny  on  how  to  get  the  most  $  for  this 
commodity. 


Discussion 

This  change  was  disseminated  much  faster  than  in  our  previous  survey,  with  a  higher  comprehension  rate.  The  open- 
ended  comments  make  it  clear  that  at  least  some  of  the  staff  in  the  field  knew  about  this  for  months,  which  is  a 
reasonable  explanation  for  both  the  high  scores  and  the  variety  of  communication  mechanisms  employed.  However, 
even  with  all  of  this  communication  over  time  there  are  still  almost  20%  of  the  respondents  who  did  not  know  any  of 
it. 

The  open-ended  comments  show  the  frustrations  of  the  DRMO  staff  with  respect  to  policy  changes  such  as  these. 
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WR  Effectiveness  Tests 


Appendix  G. 

This  appendix  includes  the  following  materials  from  our  communication  effectiveness  tests  at  WR-ALC 

•  Survey  1,  Site  Assistance  Visit  and  Operational  Readiness  Inspection,  and  its  results 

•  Survey  2,  Industrial  Prime  Vendor,  and  its  results 

•  Survey  3,  Forward  Support  Areas,  and  its  results 
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1 .  Before  receiving  this  questionnaire,  did  you  know  the  information  in  the  box  below  or  any  part  of  it? 


R1 

The  Site  Assistance  Visit  showed  that  the  implementation  of  the  AREP  process  still  has  a 
long  way  to  go. 

R2 

The  main  green  area  identified  during  the  Site  Assistance  Visit  was 
"AREP  Process  Implementation" 

R3 

The  areas  identified  as  "low"  during  the  Communications  Survey  have  all  been  resolved. 

R4 

The  main  red  area  identified  during  the  Site  Assistance  Visit  was  "Conducting  Material 
Forward  Looks". 

R5 

The  upcoming  Operational  Readiness  Inspection  (ORI)  is  an  important  event  for  WR-ALC. 

R6 

The  ORI  will  be  1-5  February. 

Please  check  only  one: 

□  No,  I  did  not  know  any  of  it. 

□  Yes,  I  knew  all  of  it. 

□  Yes,  I  knew  part  of  it.  - ►  If  so,  please  list  the  numbers  of  the  parts  you  knew. 

2.  If  you  had  the  information  in  the  box  but  you  heard  it  differently,  please  write  the  facts  you  heard  below. 


If  your  answer  above  was  "Yes,  I  knew  all  of  it, "  or "  Yes,  I  knew  part  of  it, "  please  complete  the 
questionnaire  by  providing  the  information  requested  below.  If  your  answer  above  was  "No,  I  did  not  know 
any  of  it, "  you  have  completed  the  questionnaire.  Please  return  the  questionnaire  to  me  or  drop  it  in  the 
information  box.  Thank  you  very  much  for  your  cooperation. 

3.  From  whom  did  you  first  receive  the  information  in  the  box? 

Please  check  only  one. 

□  Superiors  above  your  immediate  superior? 

□  Immediate  superiors? 

□  Peers  (others  at  your  job  level )? 

□  Subordinates? 

4.  Where  were  you  when  you  first  received  the  information  in  the  box  above? 

Please  check  only  one. 

□  I  was  working  within  my  designated  work  area  (at  my  desk,  board,  workstation,  etc.) 

□  I  was  working  within  my  department,  but  outside  of  my  designated  work  area 

□  I  was  working  outside  of  my  department 

□  I  was  on  break  outside  of  my  department,  but  still  within  the  building  (lunch,  coffee  break,  etc.) 

□  I  was  out  of  the  building  and  not  working 


Please  turn  the  page. 
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5.  How  long  ago  did  you  first  receive  the  information  in  the  box? 


Check  the  approximate  time. 

□  Today  □  1  -  2  weeks  ago 

□  1  -  2  days  ago  □  3  -  4  weeks  ago 

□  3-4  days  ago  □  5  -  6  weeks  ago 

□  5-6  days  ago 


6.  When  you  first  received  the  information  in  the  box,  did  you  believe  it? 

Please  check  only  one. 

□  I  believed  all  of  the  information 

□  I  believed  some  of  the  information 

□  I  believed  none  of  the  information 

7.  Since  you  first  received  the  information  in  the  box,  has  your  opinion  changed? 

Please  check  only  one. 

□  I  now  believe  all  of  the  information 

□  I  now  believe  some  of  the  information 

□  I  still  believe  none  of  the  information 

8.  By  what  method  did  you  first  receive  the  information  in  the  box  above? 

Please  check  only  one  of  the  following  methods. 

Written  or  visual  methods 

□  Personal  letter 

□  Letter,  memo,  or  service  program 

□  Electronic  mail  message 

□  Facsimile  (FAX) 

□  Newsletter 

□  Magazine 

□  Videotape  message 

□  Company  film 

□  Public  newspaper  or  magazine 

□  Records 

□  Our  internal  World  Wide  Web  site 

□  An  external  World  Wide  Web  site 

Talking  or  sound  methods 

□  Face-to-face  conversation 

□  Talking  over  the  telephone 

□  Talking  (and  listening)  in  a  small  group  of  two  or  more 

□  Attending  an  organized  meeting,  briefing  or  conference 

□  Overhearing  what  someone  else  said 

□  Radio  or  television 

Miscellaneous 

□  I  am  the  subject/source  of  the  message 

□  Other,  (Please  explain.)  _ _ 
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WR  Survey  1  Results.  This  section  approximates  the  results  from  printing  the  original  PowerPoint  file  using  the 
“File:  Print...:  Print  what:  Handouts  (6  slides  per  page).” 


Effectiveness  Test  #  1 

♦  6  statements 

~  4  true:  ARE?  status,  SAV  results,  ORI 
-  2  false:  Communications  survey,  ORI  date 

♦  Meetings,  held  with  all  staff  - 1  week  prior 
to  test 

♦  Email  follow-up 


Summary' 

♦  One  interviewer  collected  data  at  work  sites 

♦  4S  respondents 

♦  Analyzed  by  location 

~3?  in  Building  125.  U  “Other* 

♦  Results  similar  although 

-  125  people  knew  more 

-  Ot  hers  more  accurate  regarding  location  and 
communication  method 


How  Many  "Truo”  Options  Old  They  Know?  -  128 
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How  Many  **Tru«**  Options  Did  Th«y  Know?  ‘Othor 
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H$wMsmy’"F*r4e*  Statement*  Old  They  Correct? 

m 


How  Many  wPaS*#w  Old  ?h*y  Correct?  «- 

Other 


Coyest  <«i  5 


Tots*  Courts*  *  Other 


Total  Correct  *125 


T*»«  St Seteoted  *«<t  #»N*  $t»t*m*«t&Cis»rr»«t#d 


T<««  Statement*  S*fte**d  *fid  Fat**  5t*t»m*nt*  Corrected 


?n  formation  soars*  -  Other 


Information  Source  » 125 
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Communication  Method  *  128 


Conwittnie alien  Method  ^ Otin«r 
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Cm*;? 
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f*£-<RO*SsC«! 

$4% 


$% 


r>> 


Smalf  Gf'Sup- 
$% 


Sm«*5  Cfo«? 


&leeti»g 

13% 


fEBST 


Recommendations 

♦  More  emphasis  needs  to  he  placed  on  the 
SAV  results 

~  Try  to  connect  the  results  with  specific  impacts 
on  their  daily  tasks 

•  Remote  groups  need 

-  more  contact, 

-  more  reinforcement 

to  ensure  they  got  and  retain  the  message 


Issues  for  Next  Test 

*  False  statements  need  to  be  more  like 
‘‘misconceptions”  than  typos 

*  ERIM  will  change  administration  practices 
to  handle  false  statements,  overlapping 
response  options 

*  Need  feedback  on  results  provided 

Mbs: 
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1 .  Before  receiving  this  questionnaire,  did  you  know  the  information  in  the  box  below  or  any  part  of  it? 


Industrial  Prime  Vendor  (IPV)  is  a  test  of  best  commercial  practices  that  GAO 
R1  recommended  several  years  ago. 

R2  The  IPV  concept  is  still  evolving  and  the  scope  of  the  work  by  the  IPV  contractor  may  still 
change. 

R3  Originally,  the  IPV  contractor  was  to  take  over  all  of  the  cages,  including  Blue  Straw  and 
benchstock.  Now  IPV  just  includes  benchstock  provided  by  that  particular  contractor. 

R4  There  will  be  separate  cages  for  other  benchstock,  HAZMAT,  and  Blue  Straw  type  items. 

R5  Under  the  proposed  concept  the  mechanic  will  now  have  to  go  to  two  cages  instead  of  one. 


Please  check  only  one: 

□  No,  I  did  not  know  any  of  it. 

□  Yes,  I  knew  all  of  it. 

□  Yes,  I  knew  part  of  it.  - ►  If  so,  please  list  the  numbers  of  the  parts  you  knew. 

2.  If  you  had  the  information  in  the  box  but  you  heard  it  differently,  please  write  the  facts  you  heard  below. 


If  your  answer  above  was  "Yes,  I  knew  all  of  it, "  or "  Yes,  I  knew  part  of  it, "  please  complete  the 
questionnaire  by  providing  the  information  requested  below. 

If  your  answer  above  was  "No,  I  did  not  know  any  of  it, "  you  have  completed  the  questionnaire.  Please 
return  the  questionnaire  to  me.  Thank  you  very  much  for  your  cooperation. 

3.  From  whom  did  you  first  receive  the  information  in  the  box? 

Please  check  only  one. 

□  Superiors  above  your  immediate  superior? 

□  Immediate  superiors?  ,2 

□  Peers  (others  at  your  job  level )?  .3 

□  Subordinates?  _4 

□  None  of  the  above. 

4.  Where  were  you  when  you  first  received  the  information  in  the  box  above? 

Please  review  all  of  the  options  and  select  the  statement  that  best  represents  your  location. 

Pjease  check  only  one. 

□  I  was  working  within  my  designated  work  area  (at  my  desk,  board,  workstation,  etc.) 

□  I  was  working  within  my  department,  but  outside  of  my  designated  work  area  .2 

□  I  was  working  outside  of  my  department  .3 

□  I  was  on  break  outside  of  my  department,  but  still  within  the  building  (lunch,  coffee  break,  etc.)  ^ 

□  I  was  out  of  the  building  and  not  working  ,5 

Please  turn  the  page. 


115 


5.  How  long  ago  did  you  first  receive  the  information  in  the  box? 

Check  the  approximate  time 

□  Todays  □  1 

□  1  -  2  days  ago  .2  □  3 

□  3  -  4  days  ago  _3  □  5 

□  5  -  6  days  ago  -4 

6.  When  you  first  received  the  information  in  the  box,  did  you  believe  it? 

Please  check  only  one. 

□  I  believed  all  of  the  information 

□  I  believed  some  of  the  information  _2 

□  I  believed  none  of  the  information  ,3 

7.  Since  you  first  received  the  information  in  the  box,  has  your  opinion  changed? 

Please  check  only  one. 

□  I  now  believe  all  of  the  information 

□  I  now  believe  some  of  the  information  .2 

□  I  now  believe  none  of  the  information  .3 

8.  By  what  method  did  you  first  receive  the  information  in  the  box  above? 

Please  review  all  the  options  and  check  the  one  that  best  represents  how  you  received  this  information. 
Please  check  only  one. 

Written  or  visual  methods 

□  Personal  letter  .1 

□  Letter,  memo,  or  service  program  .2 

□  Electronic  mail  message  .3 

□  Facsimile  (FAX)  ^ 

□  Newsletter  .5 

□  Magazine  _e 

□  Videotape  message  .7 

□  Company  film  ^ 

□  Public  newspaper  or  magazine  .9 

□  Records  .10 

□  Our  internal  World  Wide  Web  site  .n 

□  An  external  World  Wide  Web  site  ,12 

Talking  or  sound  methods 

□  Face-to-face  conversation  _13 

□  Talking  over  the  telephone  .14 

□  Talking  (and  listening)  in  a  small  group  of  two  or  more  .15 

□  Attending  an  organized  meeting,  briefing  or  conference  .16 

□  Overhearing  what  someone  else  said  .17 

□  Radio  or  television  .18 

Miscellaneous 

□  I  am  the  subject/source  of  the  message  .19 

□  Other,  (Please  explain.)  .20 


-  2  weeks  ago  ,5 

-  4  weeks  ago  ^ 

-  6  weeks  ago  .7 


116 


WR-ALC  Communication  Effectiveness  Survey  #2 
Industrial  Prime  Vendor  (IPV) 

ERIM 

May  24, 1999 


During  the  week  of  May  17,  1999,  a  series  of  meetings  were  held  with  the  staff  of  the  C-141  Weapon  Systems 
Support  Center  (WSSC)  at  Wamer-Robins  Air  Logistics  Center  (WR-ALC).  Mr.  Abe  Banks,  C-141  WSSC  Chief, 
called  these  meetings  to  inform  his  staff  about  the  status  and  plans  for  the  Industrial  Prime  Vendor  (IPV)  contract. 

Most  of  these  meetings  were  held  in  Building  125.  (One  meeting  was  conducted  for  a  group  in  their  outlying 
building.)  After  the  meetings,  the  attendees  were  asked  to  complete  an  eight-question  survey  intended  to  measure  of 
the  effectiveness  of  the  management  communication  about  this  topic. 

Results 

These  results  are  based  on  62  respondents.  Overall,  this  is  a  62%  response  rate.  This  is  excellent  for  a  survey  of  this 
type. 

The  first  question  includes  five  statements  about  the  IPV  contract: 

Rl.  Industrial  Prime  Vendor  (IPV)  is  a  test  of  best  commercial  practices  that  GAO  recommended  several  years  ago. 
R2.  The  IPV  concept  is  still  evolving  and  the  scope  of  the  work  by  the  IPV  contractor  may  still  change. 

R3.  Originally,  the  IPV  contractor  was  to  take  over  all  of  the  cages,  including  Blue  Straw  and  benchstock.  Now  IPV 
just  includes  benchstock  provided  by  that  particular  contractor. 

R4.  There  will  be  separate  cages  for  other  benchstock,  HAZMAT,  and  Blue  Straw  type  items. 

R5.  Under  the  proposed  concept,  the  mechanic  will  now  have  to  go  to  two  cages  instead  of  one. 


All  of  these  statements  are  "facts"  about  the  current  state  of  the  contract.  Of  the  62  respondents,  18  said  they  had 
heard  none  of  them.  (The  interviewer  was  subjected  to  some  comments  and  criticisms  by  potential  respondents,  who 
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led  him  to  believe  that  some  of  these  18  intentionally  chose  that  option  as  a  protest.)  As  you  can  see  from  the  chart 
above,  the  other  42  respondents  "got  the  message"  quite  well. 


In  this  case,  we  know  that  the  respondents  had  just  received  the  briefing  from  the  WSSC  Chief  that  contained  the 
information  in  question.  Therefore,  you  would  expect  that  since  most  do  not  work  directly  for  Mr.  Banks,  they 
would  have  heard  the  message  from  a  superior  above  their  immediate  superior.  This  is  indeed  the  case. 


Most  of  the  meetings  were  held  in  Building  125,  the  work  area  of  most  of  the  WSSC  staff.  Some  came  to  the  meeting 
from  other  buildings  that  were  out  of  their  department  and  one  group  attended  a  meeting  held  in  their  building. 


Physical  Location 


Out  of 
Department - 
9% 


On  Break 
0% 


Not  Working 
2% 


In  Department1 
41% 


In  Work  Area 
48% 


The  questionnaires  were  completed  immediately  after  the  meeting  that  included  the  IPV  information.  Thus,  you 
would  expect  most  people  to  say  they  received  the  information  "Today".  However,  more  than  10  people  said  they 
first  heard  the  information  5-6  weeks  ago.  Since  the  IPV  was  discussed  over  a  long  period,  this  could  be  expected. 


When  Did  They  Hear  It? 


Today  1-2  Days  3-4  Days  5-6  Days  1-2  34  5-6 

Ago  Ago  Ago  Weeks  Weeks  Weeks 

Ago  Ago  Ago 


Since  trust  (or  lack  of  it)  was  a  big  issue  raised  during  our  communications  baselining,  we  also  include  two  questions 
about  believability,  i.e.,  did  you  believe  the  information  when  you  first  heard  it  and  do  you  believe  it  now?  In  this 
case,  most  of  the  respondents  just  heard  the  information.  You  would  not  expect  there  to  be  much  difference  in  the 
answers  to  these  two  questions. 


Believed  At  First 


Believed  None 


59% 


As  you  can  see  when  comparing  these  two  charts,  that  is  the  case. 

Finally,  most  staff  member  should  have  heard  the  IPV  information  in  an  organized  meeting.  We  know  from  past 
surveys,  however,  that  respondents  often  choose  the  first  option  that  resembles  their  situation.  For  example,  these 
meetings  were  "face-to-face"  and  held  in  small  groups,  making  those  options  possible.  (We  specifically  changed  the 
instructions  to  this  question  to  focus  on  this  problem,  i.e.,  "Please  review  all  the  options  and  check  the  one  that  best 
represents  how  you  received  the  information.  Please  check  only  one."  Clearly,  the  new  instructions  were  not 
enough.) 
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Believed  When  Asked 


Believe  None 

5%  Believe  All 


69% 


Communication  Method 

Overheard 

Letter,  Memo 

2%  \ 

\ 

J?  _  Email 

Meeting  jdtm 

liisr  2% 

31% 

f  A 

Small  GroupX'C__ 

y'Face-to-Face 

— 51% 

7% 

In  this  survey,  there  are  two  opportunities  for  respondents  to  provide  input  in  their  own  words: 

•  Question  2  (“did  you  hear  the  information  in  question  differently?”) 

•  Question  8  (“By  what  other  means  did  you  first  receive  the  information  in  question?”). 

For  completeness,  we  include  those  comments  modified  only  for  spelling  and  to  remove  comments  that  could 
identify  specific  respondents. 


Comments  in  Question  2 

In  this  question,  respondents  were  allowed  to  comment  on  the  “facts”  listed  in  the  first  question. 
Only  that  we  were  moving  to  a  new  location.  That  we  would  share  our  resources  with  a  contractor. 
Depends  on  who  you  talked  to,  /  heard  different  versions  of  the  information  /  heard  today. 


Even  with  the  lecturette  by  speaker -  sound-bit  lack  uniqueness  to  make  whatever  subject  was  retainable. 


Comments  in  Question  8 

Respondents  who  answered  this  question  "Other"  listed  these  information  sources. 
After  overhearing  IPV,  I  asked  my  husband  who  works  in  the  area 


Discussion 

The  circumstances  of  this  data  collection  were  less  than  ideal.  Providing  a  questionnaire  asking  about  a  just 
completed  briefing  must  have  seemed  like  a  quiz.  It  is  not  surprising  that  our  researcher  received  some  verbal  abuse. 

In  spite  of  this  issue,  those  42  staff  members  who  did  respond  seem  to  understand  the  status  of  IPV  quite  well.  Since 
this  is  an  on-going  issue  (note  the  large  number  of  respondents  who  first  heard  about  this  5-6  weeks  before),  it  is 
important  to  keep  the  staff  up-to-date  with  the  latest  changes.  Sources  of  misinformation  are  readily  available.  Low 
staff  morale  (evident  by  the  nature  of  the  comments  made  to  our  researcher)  makes  it  easier  to  leap  upon  new 
information  not  coming  from  formal  channels  as  yet  another  strike  against  management. 
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1 .  Before  receiving  this  questionnaire,  did  you  know  the  information  in  the  box  below  or  any  part  of  it? 


R1  The  completion  of  the  Forward  Support  Areas  (FSAs)  is  another  step  toward  fully 
implementing  AREP. 

R2  The  tail  team  concept  will  be  used  to  determine  seating  arrangements. 

R3  Since  supervisory  and  non-supervisory  people  will  be  sitting  together,  a  lot  of  cooperation 
and  consideration  will  be  required. 

R4  We  will  continue  to  have  formal  tail  team  meetings. 

R5  Having  the  ALSs  and  production  sit  close  together  will  improve  communications  between 
them.  _ _ _ 


Please  check  only  one: 

□  No,  I  did  not  know  any  of  it. 

□  Yes,  I  knew  all  of  it. 

□  Yes,  I  knew  part  of  it.  - *  If  so,  please  list  the  numbers  of  the  parts  you  knew. 

2.  If  you  had  the  information  in  the  box  but  you  heard  it  differently,  please  write  the  facts  you  heard  below. 


If  your  answer  above  was  "Yes,  I  knew  all  of  it, "  or "  Yes,  I  knew  part  of  it, "  please  complete  the 
questionnaire  by  providing  the  information  requested  below. 

If  your  answer  above  was  "No,  I  did  not  know  any  of  it, "  you  have  completed  the  questionnaire.  Please 
return  the  questionnaire  to  me.  Thank  you  very  much  for  your  cooperation. 

3.  From  whom  did  you  first  receive  the  information  in  the  box? 

Please  check  only  one. 

□  Superiors  above  your  immediate  superior?  .i 

□  Immediate  superiors?  .2 

□  Peers  (others  at  your  job  level )?  .3 

□  Subordinates?  _4 

□  None  of  the  above.  _s 

4.  Where  were  you  when  you  first  received  the  information  in  the  box  above? 

Please  review  all  of  the  options  and  select  the  statement  that  best  represents  your  location. 

Pjease  check  only  one. 

□  I  was  working  within  my  designated  work  area  (at  my  desk,  board,  workstation,  etc.) 

□  I  was  working  within  my  department,  but  outside  of  my  designated  work  area  .2 

□  I  was  working  outside  of  my  department  .3 

□  I  was  on  break  outside  of  my  department,  but  still  within  the  building  (lunch,  coffee  break,  etc.)  .4 

□  I  was  out  of  the  building  and  not  working  .5 

Please  turn  the  page. 
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5.  How  long  ago  did  you  first  receive  the  information  in  the  box? 

Check  the  approximate  time. 

□  Today.!  □  1  - 2  weeks  ago ,5 

□  1  -  2  days  ago  .2  □  3  -  4  weeks  ago  ^ 

□  3  -  4  days  ago  ,3  □  5  -  6  weeks  ago  ,7 

□  5  -  6  days  ago  ^ 

6.  When  you  first  received  the  information  in  the  box,  did  you  believe  it? 

Please  check  only  one. 

□  I  believed  all  of  the  information  .i 

□  I  believed  some  of  the  information  .2 

□  I  believed  none  of  the  information  .3 

7.  Since  you  first  received  the  information  in  the  box,  has  your  opinion  changed? 

Please  check  only  one. 

□  I  now  believe  all  of  the  information 

□  I  now  believe  some  of  the  information  ,2 

□  I  now  believe  none  of  the  information  .3 

8.  By  what  method  did  you  first  receive  the  information  in  the  box  above? 

Please  review  all  the  options  and  check  the  one  that  best  represents  how  you  received  this  information. 
Please  check  only  one. 

Written  or  visual  methods 

□  Personal  letter 

□  Letter,  memo,  or  service  program  .2 

□  Electronic  mail  message  .3 

□  Facsimile  (FAX)  ^ 

□  Newsletter  .5 

□  Magazine  ^ 

□  Videotape  message  .7 

□  Company  film  ^ 

□  Public  newspaper  or  magazine  ,9 

□  Records  _10 

□  Our  internal  World  Wide  Web  site  .n 

□  An  external  World  Wide  Web  site  .12 

Talking  or  sound  methods 

□  Face-to-face  conversation  _13 

□  Talking  over  the  telephone  ,14 

□  Talking  (and  listening)  in  a  small  group  of  two  or  more  .15 

□  Attending  an  organized  meeting,  briefing  or  conference  .16 

□  Overhearing  what  someone  else  said  .17 

□  Radio  or  television  .i8 

Miscellaneous 

□  I  am  the  subject/source  of  the  message  ,i9 

□  Other,  (Please  explain.)  ,2o 
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WR-ALC  Communication  Effectiveness  Survey  #3 
Forward  Support  Areas  (FSAs) 

ERIM 

May  24, 1999 


During  the  week  of  May  17,  1999,  a  series  of  meetings  were  held  with  the  staff  of  the  C-141  Weapon  Systems 
Support  Center  (WSSC)  at  Wamer-Robins  Air  Logistics  Center  (WR-ALC).  Mr.  Abe  Banks,  C-141  WSSC  Chief, 
called  these  meetings  to  inform  his  staff  about  the  status  and  plans  for  the  Forward  Support  Areas  (FSAs)  to  be 
implemented  in  support  of  the  WSSCs  activities. 

Most  of  these  meetings  were  held  in  Building  125.  (One  meeting  was  conducted  for  a  group  in  their  outlying 
building.)  After  the  meetings,  the  attendees  were  asked  to  complete  an  eight-question  survey  intended  to  measure  of 
the  effectiveness  of  the  management  communication  about  this  topic. 


Results 

These  results  are  based  on  63  respondents.  Overall,  this  is  a  63%  response  rate.  This  is  excellent  for  a  survey  of  this 
type. 

The  first  question  includes  five  statements  about  the  Forward  Support  Areas: 

Rl.  The  completion  of  the  Forward  Support  Areas  (FSAs)  is  another  step  toward  fully  implementing  AREP. 

R2.  The  tail  team  concept  will  be  used  to  determine  seating  arrangements. 

R3.  Since  supervisory  and  non-supervisory  people  will  be  sitting  together,  a  lot  of  cooperation  and  consideration 
will  be  required. 

R4.  We  will  continue  to  have  formal  tail  team  meetings. 

R5.  Having  the  ALSs  and  production  sit  close  together  will  improve  communications  between  them. 


Which  Statements  Did  They  Choose? 


All  of  these  statements  are  "facts"  about  the  current  state  of  the  contract.  Of  the  63  respondents,  20  said  they  had 
heard  none  of  them.  (The  interviewer  was  subjected  to  some  comments  and  criticisms  by  potential  respondents,  who 
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led  him  to  believe  that  some  of  these  20  intentionally  chose  that  option  as  a  protest.)  As  you  can  see  from  the  chart 
above,  the  other  42  respondents  "got  the  message"  quite  well. 

In  this  case,  we  know  that  the  respondents  had  just  received  the  briefing  from  the  WSSC  Chief  that  contained  the 
information  in  question.  Therefore,  you  would  expect  that  since  most  do  not  work  directly  for  Mr.  Banks,  they 
would  have  heard  the  message  from  a  superior  above  their  immediate  superior.  This  is  clearly  the  case,  although 
many  people  also  are  hearing  information  about  this  topic  "through  the  grapevine"  from  their  peers  and  other  sources 
not  reflected  in  our  question. 


Most  of  the  meetings  were  held  in  Building  125,  the  work  area  of  most  of  the  WSSC  staff.  Some  came  to  the  meeting 
from  other  buildings  that  were  out  of  their  department  and  one  group  attended  a  meeting  held  in  their  building. 


Physical  Location 

On  Break 

2%  Not  Working 

Out  of 

\  /  2% 

Department-- 

- . 

6% 

Work  Area 

/ 

64% 

In  Department 

26% 

The  questionnaires  were  completed  immediately  after  the  meeting  that  included  the  FSA  information.  Thus,  you 
would  expect  most  people  to  say  they  received  the  information  "Today".  However,  15  people  said  they  first  heard 
the  information  5-6  weeks  ago.  Since  the  FSAs  were  discussed  over  a  long  period,  this  could  be  expected. 
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Since  trust  (or  lack  of  it)  was  a  big  issue  raised  during  our  communications  baselining,  we  also  include  two  questions 
about  believability,  i.e.,  did  you  believe  the  information  when  you  first  heard  it  and  do  you  believe  it  now?  In  this 
case,  most  of  the  respondents  just  heard  the  information.  You  would  not  expect  there  to  be  much  difference  in  the 
answers  to  these  two  questions. 


Believed  At  First 


Believed  None 
13% 


51% 


As  you  can  see  when  comparing  these  two  charts,  that  is  not  the  case.  Perhaps  this  briefing  cleared  up  some  of  the 
misconceptions. 

Finally,  most  staff  member  should  have  heard  the  FSA  information  in  an  organized  meeting.  We  know  from  past 
surveys,  however,  that  respondents  often  choose  the  first  option  that  resembles  their  situation.  For  example,  these 
meetings  were  "face-to-face”  and  held  in  small  groups,  making  those  options  possible.  (We  specifically  changed  the 
instructions  to  this  question  to  focus  on  this  problem,  i.e.,  "Please  review  all  the  options  and  check  the  one  that  best 
represents  how  you  received  the  information.  Please  check  only  one."  Clearly,  the  new  instructions  were  not 
enough.) 
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In  this  survey,  there  are  two  opportunities  for  respondents  to  provide  input  in  their  own  words: 

•  Question  2  (“did  you  hear  the  information  in  question  differently?”) 

•  Question  8  (“By  what  other  means  did  you  first  receive  the  information  in  question?”). 

For  completeness,  we  include  those  comments  modified  only  for  spelling  and  to  remove  comments  that  could 
identify  specific  respondents. 

Comments  in  Question  2 

In  this  question,  respondents  were  allowed  to  comment  on  the  “facts”  listed  in  the  first  question. 

Even  with  the  lecturette  by  speaker -  sound-bit -  NO 
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Comments  in  Question  8 

The  one  respondent  who  answered  this  question  "Other"  did  not  list  their  information  source. 


Discussion 

The  circumstances  of  this  data  collection  were  less  than  ideal.  Providing  a  questionnaire  asking  about  a  just 
complettdbriefing  must  have  seemed  like  a  quiz.  I.  is  no.  apprising  tha,  our  researcher  rece.ved  some  verbal  abuse. 

In  spite  of  this  issue,  those  43  staff  members  who  did  respond  seem  to  understand  the  status  of. the  F. o™rc I  Support 
Areas  (FSAs)  quite  well.  Since  this  is  an  on-going  issue  (note  the  large  number  of  respondents  who  first  heard  about 
ads  5-6  weeks  before),  it  is  important  to  keep  the  staff  up-to-date  with  the  latest  changes.  Sources  of  misinformation 
are  readily  available.  Low  staff  morale  (evident  by  the  nature  of  the  comments  made  to  our  researcher)  makes  it 
easier  to  leap  upon  new  information  not  coming  from  formal  channels  as  yet  another  strike  agamst  management. 
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